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Abstract 

The  United  States  Air  Force  has  emphasized  a  need  to  improve  the 
quality  of  its  cost  estimates.  It  seems  that  cost  analysts  who  are  ade- 
quately  educated  can  provide  the  high  quality  estimates  needed  by  the  Air 
Force.  This  research  effort  developed  an  educational  requirements  list 
for  Air  Force  cost  analysts.  This  list  is  needed  to  evaluate  the  educa¬ 
tional  qualifications  of  not  only  potential  cost  analysts,  but  current 
analysts  as  well.  The  list  is  based  on  an  evaluation  of  a  number  of  cost 
analysis  related  subjects  by  experienced  Air  Force  cost  analysts  via  a 
mail  survey.  Experienced  analysts  rated  thirty-five  cost  analysis  sub¬ 
jects  in  terms  of  usefulness  to  job  performance.  Analysts  were  also  asked 
to  identify  appropriate  educational  methods  for  acquiring  knowledge  in  the 
subjects.  The  result  of  this  effort  is  an  Air  Force  cost  analyst  educa¬ 
tional  requirements  list  by  educational  method. 

Analysis  of  the  survey  data  was  accomplished  utilizing  a  computer 
crosstabulation  procedure  which  compared  responses  to  each  subject.  The 
analysis  used  decision  rules  which  measured  the  central  tendency  of  the 
responses.  Based  on  the  decision  rules  outlined  in  the  methodology 
chapter  subjects  were  recommended  to  be  taught  by  various  educational 
methods.  Also,  written  comments  are  presented  on  how  experienced  cost 
analysts  perceived  the  educational  competence  of  all  Air  Force  cost  ana¬ 
lysts  can  be  improved. 
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AN  ANALYSIS  OF  THE  EDUCATIONAL  REQUIREMENTS  OF  AIR  FORCE  COST  ANALYSTS 


I.  Research  Overview 


General  Issue 

The  present  concern  of  many  government  officials  over  cost  growth  has 
increased  the  need  for  good  cost  estimating  throughout  the  Department  of 
Defense  (Margolis,  1981:1,17).  Most  recently,  the  United  States  Air  Force 
has  emphasized  the  need  to  improve  the  quality  of  its  cost  estimates. 
However,  improving  the  quality  of  these  estimates  involves  three  aspects. 
Two  currently  being  pursued  involve  the  development  of  techniques  and  the 
establishment  of  data  bases  used  to  generate  cost  estimates  (Margolis, 
1981:17).  The  other  aspect  consists  of  providing  capable  personnel  to 
generate  high  quality  estimates.  Providing  capable  personnel  means  Air 
Force  cost  analysts  must  both  be  adequately  educated  and  well  trained. 
Training  comes  through  experiences  gained  on  the  job.  Education  is 
acquired  through  formal  course  work  taken  as  part  of  a  college  education 
or  through  professional  continuing  education  (PCE).  Although  almost  all 
cost  analysts  have  a  bachelor's  degree,  the  courses  taken  may  or  may  not 
have  any  relevance  to  cost  analysis.  Consequently,  PCE  courses  may  con- 
sititute  all  or  a  significant  portion  of  an  individuals  cost  analysis 
education.  However,  there  is  no  list  of  standard  Air  Force  cost  analyst 
educational  qualifications  (Goetsch,  1980:10).  This  absence  strongly 
implies  that  the  continued  education  of  an  analyst  is  a  task  left  mainly 
to  the  individual  and  his  supervisor.  This  is  particularly  true  for  PCE 
courses.  Some  managers  may  take  an  active  role  in  educating  their  ana¬ 
lysts  by  requiring  them  to  complete  specified  courses.  However,  this 
type  of  active  participation  may  be  neglected  in  the  absence  of  a  man¬ 
datory  educational  requirements  plan. 
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Specific  Problem 


Educational  competence  is  the  body  of  knowledge  that  a  cost  analyst 
should  possess  to  perform  his  job  effectively.  This  body  of  knowledge 
contains  both  job  specific  knowledge  which  relates  to  Che  unique  require¬ 
ments  of  a  given  set  of  tasks,  and  general  knowledge  which  all  cost  ana¬ 
lysts  should  possess  in  order  to  be  considered  fully  qualified  analysts. 
This  issue  of  educational  competence  can  only  be  partialLy  satisfied  by 
an  individual  analyst.  Certainly,  the  individual  analyst  and  his  super¬ 
visor  should  decide  which  job  specific  knowledge  is  required  to  perform 
various  tasks.  However,  the  decision  concerning  which  general  knowledge 
is  required  is  too  critical  an  issue  to  be  left  for  each  analyst  to 
discern.  The  individual  or  his  supervisor  may  have  a  short  term  perspec¬ 
tive  that  is  only  job  related  and  fail  to  acknowledge  the  importance  of 
developing  overall  professional  competence.  Consequently,  the  educa¬ 
tional  plan  developed  by  the  individual  or  his  supervisor  may  contain 
major  deficiencies.  The  Air  Force  needs  an  established  standard  for 
measuring  the  adequacy  of  educational  competence  among  its  cost  analysts 
which  will  induce  a  higher  degree  of  quality  in  all  Air  Force  cost  esti¬ 
mating.  This  standard  should  identify  just  what  subjects  should  be 
offered  to  provide  the  necessary  general  knowledge.  The  general 
knowledge  should  be  distinguished  from  job  specific  knowledge  so  that 
individual  analysts  are  not  forced  to  acquire  job  specific  knowledge  not 
required  to  perform  their  current  duties.  To  prevent  this,  job  specific 
knowledge  should  only  be  offered  in  the  form  of  elective  courses  which 
the  individual  may  take  based  on  his  particular  job  requirements. 

However,  general  knowledge  should  be  offered  in  the  form  of  mandatory 
courses  required  of  all  cost  analysts.  Perhaps,  by  establishing  a 


2 


comprehensive  cost  analysis  educational  program  composed  of  mandatory 
and  elective  subjects,  a  standard  can  be  developed  which  adequately  satis¬ 
fies  the  needs  of  the  Air  Force. 

Literature  Review  and  Background 

Investigation  of  the  problem  involved  examining  the  curriculum  of  the 
cost  analysis  master's  degree  program  at  the  Air  Force  Institute  of 
Technology  (AFIT)  as  well  as  the  curriculums  of  various  undergraduate 
degree  programs  deemed  appropriate  for  the  cost  analyst.  Other  con¬ 
siderations  included  an  examination  of  the  professional  continuing  educa¬ 
tion  (PCE)  program  at  AFIT  as  it  applies  to  Air  Force  cost  analysts.  In 
addition,  Air  Force  cost  analysis  on-the-job  training  (OJT)  was  also 
examined. 

The  nature  of  the  information  required  to  evaluate  the  problem  iden¬ 
tified  by  this  research  effort  dictated  a  review  of  past  efforts  which 
examined  similar  problems.  Determining  the  adequacy  of  curriculums  is  by 
no  means  a  new  issue  to  the  Air  Force  academic  community.  In  particular, 
previous  AFIT  theses  provided  a  valuable  source  of  reference  material. 

Allan  C.  Hart  performed  a  study  in  1965  evaluating  the  utilization  of 
the  logistics  education  offered  at  that  time  by  AFIT.  Data  were  provided 
via  a  questionnaire  distributed  to  1963  and  1964  AFIT  graduates. 

Utilizing  a  hypothesis  test  approach,  Hart  concluded  that  in  addition  to 
meeting  the  institutional  objectives,  the  curriculum  of  the  School  of 
Systems  and  Logistics  provided  the  necessary  tools  for  solving  major 
logistics  and  weapon  system  problems  (Hart,  1965:6,7,57). 

In  1968  Robert  Cook  and  John  Greene  conducted  a  study  evaluating  the 
relevancy  of  the  then  current  School  of  Systems  and  Logistics  curriculum 
to  the  then  current  operational  requirements.  The  team  used  a  question- 
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naire  to  gather  the  necessary  data  to  perform  their  evaluation.  A  survey 
of  graduates  from  1965,  1966,  and  1967  revealed  that  the  AF1T  curriculum 
was  adequate  to  meet  the  needs  of  the  operational  environment  at  that  time 
(Cook,  1968:4,95). 

In  1971  a  research  effort  conducted  by  James  Cushman  and  James 
Townsend  evaluated  the  adequacy  of  courses  available  to  Air  Force  main¬ 
tenance  managers  for  providing  the  necessary  management  education  for  a 
chief  of  aircraft  maintenance  to  perform  his  duties  effectively.  Through 
the  use  of  a  questionnaire  and  a  guided  interview  the  researchers 
concluded  that  the  courses  available  to  Air  Force  maintenance  managers 
provided  adequate  management  education  in  both  production  management  and 
communicative  skills.  However,  the  courses  available  to  the  maintenance 
manager  did  not  provide  the  necessary  management  base  in  the  discipline  of 
statistics.  Only  the  three  management  disciplines  of  production  manage¬ 
ment,  communicative  skills,  and  statistics  were  evaluated  by  the 
researchers  (Cushman,  1971:12,68-69). 

In  1972  James  Ross  and  Earl  Steiner  recognized  the  continuing  need  to 
evaluate  the  AFIT  logistics  education  program.  Their  research  team  sought 
to  discover  if  the  Air  Force  Logistics  Command  (AFLC)  civilian  employees 
viewed  formal  education  as  important.  They  also  wanted  to  know  what  edu¬ 
cation  would  prepare  AFLC  civilian  employees  to  be  more  effective  managers. 
They  concluded  via  a  survey  instrument  that  education  was  extremely 
important  to  civilian  AFLC  employees.  They  also  noted  that  educational 
programs  developed  for  logistical  personnel  should  include  management, 
communicative  science,  and  personnel  management  as  major  disciplines  in 
the  curriculum  (Ross,  1972:7-8,50-52). 

In  1978  another  study  was  conducted  by  William  Crowder  and  James 


Davidson  to  evaluate  the  "usefulness"  of  the  graduate  logistics  program  at 
AFIT.  The  objective  of  the  research  effort  was  to  determine  the  extent  to 
which  logistics  graduates  were  using  their  logistics  education. 
Questionnaires  were  distributed  to  graduates  of  the  1971A  through  1975B 
classes  and  their  supervisors  to  gather  data.  Analysis  of  the  data 
revealed  that  both  the  graduates  and  their  supervisors  believed  that  the 
AFIT  logistics  program,  was  useful  (Crowder,  1978:1,9,11,56). 

Since  the  AFIT  master's  program  in  cost  analysis  has  its  foundation  in 
the  system  management  program  a  more  recent  study  conducted  by  Ernest 
Speck  i3  of  particular  importance  to  this  research  effort.  In  1981  Speck 
was  involved  in  an  effort  to  determine  the  usefulness  of  the  AFIT  Graduate 
Systems  Management  program  to  Air  Force  Officers  graduating  in  the  classes 
of  1969  through  1978  (Speck,  1981:2).  Utilizing  a  questionnaire  to 
gather  data  on  the  issue,  Speck  concluded  that  the  AFIT  Graduate  Systems 
Management  program  sufficiently  supported  the  need  of  its  graduates. 
However  the  graduates  indicated  a  need  for  more  emphasis  on  practical 
applications  and  less  theoretical  emphasis  (Speck,  1981:58-60). 

Another  item  of  significant  importance  to  this  research  effort  is  a 

collection  of  papers  by  Wayne  M.  Allen,  the  Director  of  Cost  Analysis, 

Office  of  the  Comptroller  of  the  Army.  In  this  collection  of  papers 

entitled  "Towards  a  Cost  Analysis  Career  (Job)  Series,"  Allen  builds  a 

strong  case  towards  establishing  a  unique  defense  job  series  for  Army 

cost  analysts.  In  the  preface  of  the  document  Allen  states: 

This  compendium  of  papers  chronicles  the  movement  within 
the  Army  to  improve  the  cost  estimating  and  analysis 
process  through  recognizing  the  "people  element" — the  need 
for  improved  mechanisms  for  identifying,  selecting,  training, 
transferring,  evaluating,  promoting,  and  retaining  cost 
analysts.  It  is  dedicated  to  all  who  have  interests  along 
these  lines  (Allen,  1980). 
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purposes.  Also,  those  questionnaires  in  which  the  respondent  did  not 
identify  at  least  four  years  experience  in  the  cost  analysis  field  were 

also  omitted. 

The  data  provided  bv  the  questionnaire  falls  in  the  categories  of  nomi¬ 
nal  and  ordinal  with  the  usefulness  rating  representing  an  ordinal  scale  of 
measurement  and  the  educational  methods  representing  a  nominal  measurement 
scale.  Emory  describes  the  various  scales  of  measurement  as  follows: 

Type  of  Scale  Characteristics  of  Scale 

Nominal  No  order,  distance  or  origin. 

Ordinal  Order,  but  no  distance  or  unique 

origin . 

Interval  Both  order  and  distance  but  no 

unique  origin. 

Ratio  Order,  distance,  and  unique 

origin  (Emory,  1980:121). 

Ordinal  scales  are  used  to  establish  rank,  order  among  alternatives  with  no 
particular  meaning  assigned  to  the  differences  among  ranks.  Ordinal  scales 
only  differentiate  "better"  from  "worse".  The  appropriate  measure  of 
central  tendency  for  ordinal  data  is  the  median.  The  median  is  the  middle 
value  of  a  rank-ordered  data  set.  Half  the  values  in  the  data  set  will  lie 
below  the  median,  and  half  the  values  will  lie  above  the  median.  Nominal 
scales  are  used  to  identify  the  class  or  category  to  which  an  observation 
belongs.  The  appropriate  measure  of  central  tendency  is  the  mode,  which 
identifies  the  class  or  category  containing  the  largest  number  of  obser¬ 
vations  (Emory,  1980:122-124). 

Contingency  tables  were  used  to  compare  the  usefulness  rating  and  edu¬ 
cational  methods  identified  by  respondents.  The  tables  provided  frequency 
counts  by  rows,  columns  and  total  responses.  The  Crosstabs  procedure 


The  questionnaire  was  evaluated  by  several  AFIT  instructors  for  comple¬ 
teness  and  suitability.  In  addition,  Captain  Ben  Dilla  of  the  AFIT 
Organization  Sciences  Department  provided  some  helpful  suggestions  on  the 
format  of  the  questionnaire.  A  short  response  time  increased  the  likeli¬ 
hood  that  members  of  the  population  would  .eturn  the  questionnaire. 

Therefore,  a  goal  was  established  that  the  questionnaire  not  take  longer 
than  twenty  minutes  to  complete.  In  order  to  estimate  the  required 
response  time,  three  AFIT  faculty  members  were  asked  to  complete  the 
questionnaire.  It  was  estimated  that  the  questionnaire  would  take  approxi¬ 
mately  ten  minutes  to  complete,  a  time  well  within  the  established  goal.  A 
copy  of  the  questionnaire  can  be  found  in  Appendix  A  of  this  report. 

Population  Background 

The  questionnaire  was  distributed  Air  Force  wide.  The  Air  Force  Military 
Personnel  Center  (AFPMC)  and  the  Cost  and  Management  Analysis  Directorate 
at  Headquarters  United  States  Air  Force  provided  a  current  list  of  cost 
analysts  located  in  the  continental  United  States  (CONUS).  A  question¬ 
naire  was  sent  to  each  analyst  in  the  population.  Of  course,  there  was  no 
way  of  assuring  the  return  of  each  questionnaire.  However,  an  effort  was 
made  to  encourage  a  response  from  each  of  analysts  in  the  population.  A 
letter  from  Colonel  Pfluram  the  Director  of  Cost  and  Management  Analysis 
at  Headquarters  Air  Force  (AF/ACM),  accompanied  the  questionnaire  to 
express  the  concern  for  individual  responses. 

Analysis  Description 

Each  returned  questionnaire  was  screened  to  determine  if  it  was 
compLete  and  all  responses  were  correctly  recorded.  Those  questionnaires 
considered  incomplete  or  improperly  filled  out  were  omitted  for  data 
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represented  "extremely  useful,"  "useful,"  and  "of  little  use"  respec¬ 
tively.  A  response  of  "critical"  would  imply  that  the  respondent  felt 
that  this  subject  should  be  taught  to  every  analyst.  A  response  of 
"useless"  would  imply  that  a  cost  analyst  has  no  need  to  know  anything 
about  this  particular  subject.  A  response  of  "useful"  reflects  the  opin¬ 
ion  that  while  this  subject  is  of  value,  an  analyst  could  probably  per¬ 
form  adequately,  but  at  a  lower  level  of  competence,  without  this 
knowledge.  The  responses  "extremely  useful"  and  "of  little  use"  were  used 
to  allow  the  respondent  some  flexibility  in  rating  the  subjects. 

Analysts  were  also  asked  to  identify  which  of  four  educational  methods 
were  deemed  appropriate  for  acquiring  the  knowledge  in  the  subjects  they 
rated.  The  four  educational  methods  were: 

1.  On-the-job  training  (OJT) 

2.  Professional  continuing  education  (PCE) 

3.  Undergraduate  education  (UG) 

4.  Graduate  education  (GRAD) 

Respondents  were  allowed  to  choose  multiple  combinations.  They  could 
select  one  of  the  four  methods,  any  one  of  six  possible  combinations  of 
two  methods,  any  one  of  four  possible  combinations  of  three  methods,  or 
even  all  four  methods.  This  allowed  for  fifteen  possible  responses. 

In  addition,  analysts  were  asked  to  rate  several  PCE  courses  offered 
by  AFIT  using  the  same  five-point  scale.  The  courses  evaluated  were: 

1.  QMT  170  Principles  of  Contract  Pricing 

2.  QMT  180  Learning  Curve  Analysis 

3.  QMT  345  Introductory  Quantitative  Analysis 

4.  QMT  353  Introduction  to  Life  Cycle  Cost  Management 

5.  QMT  540  Advanced  Pricing  Methods 

6.  QMT  550  Advanced  Quantitative  Methods 

7.  QMT  551  Advanced  Cost  and  Economic  Analysis 

8.  SYS  362  Cost  Schedule  Control  System  Criteria 

As  a  final  task  each  analyst  was  asked  to  briefly  describe  how  the  educ 
tional  competence  could  be  improved  for  all  Air  Force  cost  analysts. 


analysis.  Also  along  chose  lines,  ideas  concerning  undergraduate  degree 
preferences  varied  across  the  spectrum.  Degree  preferences  varied  from 
English  to  chemical  engineering.  There  also  seemed  to  be  a  general  con¬ 
sensus  among  several  who  were  interviewed  to  place  personality  and  atti¬ 
tude  above  education  in  predicting  successful  job  performance.  Moreover, 
all  tended  to  agree  that  at  least  four  years  experience  in  the  cost  ana¬ 
lysis  field  was  necessary  to  gain  a  general  working  knowledge  for  effec¬ 
tive  job  performance. 


Questionnaire  Development  and  Format  Description 

After  the  interviews  were  completed,  a  comparison  was  made  between  the 
headings  contained  in  the  interview  guide  and  the  subjects  identified 
during  the  interviews.  A  review  of  AF1T,  Wright  State,  Ohio  State,  and 
University  of  Dayton  catalogs  revealed  a  number  of  other  cost  analysis 
related  subjects.  This  analysis  resulted  in  the  identif ication  of  twenty- 
seven  subjects  which  were  potential  candidates  for  a  cost  analysis  educa¬ 
tional  program.  These  subjects  are  listed  below: 


1.  Basic  Programming 

2.  Calculus 

3.  Cobol  Programming 

4.  Defense  Production  Management 

5.  Federal  Financial  Management 

6.  Financial  and  Cost  Accounting 

7.  Fortran  Programming 

8.  Introductory  Statistics 

9.  Linear  Mathematical  Models 

10.  Macroeconomics 

11.  Mgt  and  Behavior  in  Organizations 

12.  Management  Information  Systems 

13.  Managerial  Accounting 

14.  Managerial  Finance 


15.  Managerial  Statistics 

16.  Matrices  and  Linear  Algebra 

17.  Microeconomics 

18.  Production  Management 

19.  Quantitative  Analysis 

20.  Quantitative  Decisionmaking 

21.  Regression  I 

22.  Regression  II 

23.  R  &  D  Management 

24.  Research  Methods 

25.  Seminar  in  Cost  Analysis 

26.  Technical  Communications 

27.  Technological  Forecasting 


Experienced  Air  Force  cost  analysts  were  asked  to  rate  the  twenty- 
seven  subjects  on  a  scale  from  one  to  five  with  one  being  "critical" 
to  the  job  and  five  being  "useless."  The  numbers  two,  three,  and  four 
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Interview  Results 


The  interviews  began  during  the  latter  weeks  of  February  1984  and  were 
completed  during  the  first  week  of  March  1984.  The  nine  interviews  con¬ 
ducted  ranged  from  thirty  minutes  to  one  hour  in  length.  All  of  the  indi¬ 
viduals  identified  seemed  interested  in  the  research  and  were  more  than 
willing  to  participate.  Each  analyst  interviewed  was  asked  to  identify 
educational  requirements,  based  upon  the  general  headings  contained  in  the 
interview  guide,  that  he  felt  were  essential  to  the  effective  performance 
of  an  Air  Force  cost  analyst.  A  number  of  subjects  were  discussed  under 
each  of  the  general  headings,  and  the  identification  of  these  subjects  was 
quite  helpful  in  developing  the  questionnaire  that  followed.  There  were 
several  subjects  that  were  identified  under  the  "OTHER  TOPICS"  area  which 
were  considered  quite  important  and  may  have  been  otherwise  overlooked. 

Those  subjects  were: 

Logistics  Management 

R  &  D/Systems  Management 

Contract  Administration/Acquisition  Management 

Production  Management 

Defense  Contractor  and  Air  Force  Relations 

Before  mailing  the  respondents  final  questionnaire  packages,  a  com¬ 
parison  was  made  between  the  questionnaire  and  the  interview  results. 
Engineering  was  a  general  area  which  was  also  identified  as  important,  but 
none  of  those  interviewed  would  identify  any  specific  type  of  engineering 
as  being  more  important  that  other  types.  An  engineering  background  of  a 
"general  nature"  was  all  that  they  would  identify,  insisting  that  such  a 
background  would  be  extremely  helpful  in  communicating  with  engineers  when 
making  inquiries.  Several  individuals  pointed  out  that  it  was  good  to 
have  a  cost  analysis  team  with  varied  backgrounds  to  achieve  a  comprehensive 
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at  Wright-Patterson  Air  Force  Base.  Five  of  the  interviews  were  with 
individuals  from  Air  Force  Systems  Command  (AFSC),  Aeronautical  System 
Division’s  Cost  Analysis  Directorate  (ASD/ACC).  The  other  four  were  with 
individuals  from  Air  Force  Logistics  Command  (AFLC)  Headquarter's  Cost  and 
Management  Analysis  Directorate  (AFLC/ACM).  These  individuals  were  chosen 
for  three  reasons.  First,  both  commands  were  represented  by  a  significant 
number  of  highly  experienced  analysts,  with  experience  levels  ranging  from 
four  to  twenty  years.  Second,  the  analysts  possessed  varied  backgrounds 
with  their  formal  education  including  everything  from  anthropology  to  phy¬ 
sics.  Also  the  extensive  experiences  of  the  those  interviewed  aided  in 
establishing  the  comprehensiveness  of  the  questionnaire.  Third,  the  anal¬ 
ysts  from  the  two  commands  represented  a  contrast  in  viewpoints  which  was 
extremely  useful  in  comparing  opinions.  Since  many  of  the  cost  analysts 
in  AFSC  and  AFLC  are  located  at  Wright-Patterson,  the  convenience  in 
accessing  the  analysts  was  also  a  prime  consideration. 

It  was  decided  that  an  interview  guide  would  be  useful  in  establishing 
the  content  of  the  questionnaire  by  helping  to  guide  the  discussion  and 
act  as  a  prompt  to  stimulate  ideas  during  the  interview.  The  interview 
guide  consisted  of  a  list  of  topic  questions  or  areas  identified  through 
the  literature  review  as  being  applicable  to  the  cost  analysis  field.  The 
expectation  was  that  this  guide  would  lead  the  analysts  to  identify 
omitted  topics  or  areas  of  concern.  This  step  was  taken  to  insure  that 
the  questionnaire,  which  was  the  primary  survey  instrument,  was  adequately 
evaluated.  In  addition,  the  interview  guide  helped  to  minimize  the  poten¬ 
tial  influence  that  the  researcher's  attitudes,  opinions  and  biases  may 
have  had  on  the  analysts.  Furthermore,  the  analysts  were  guaranteed  ano¬ 
nymity  to  induce  candid  responses  during  the  interviews.  The  full  text  of 
the  interview  guide  is  listed  in  Appendix  A. 
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research  questions  identified  in  chapter  one  required  solicitation  of  opi¬ 
nions  from  a  population  of  nearly  four  hundred  geographically  dispersed 
cost  analysts.  Therefore,  the  questionnaire  was  the  only  practical  means 
of  acquiring  the  needed  information.  In  order  to  develop  and  validate  the 
content  of  the  questionnaire,  however,  it  was  necessary  to  rely  on  the 
expert  opinions  of  experienced  cost  analysts.  Soliciting  these  opinions 
required  a  method  of  exchange  which  allowed  free  expression  and  crossflow 
of  information  between  the  researcher  and  the  analyst.  This  could  best  be 
accomplished  by  means  of  a  personal  interview.  Thus,  it  was  necessary  to 
explore  the  strengths  and  weaknesses  of  both  the  interview  and  the 
questionnaire  process. 

The  primary  strength  of  the  questionnaire,  as  previously  mentioned,  is 
its  versatility  in  gathering  needed  data.  Questioning  is  the  only  way  of 
getting  information  on  private  behavior,  attitudes,  or  opinions.  The 
questionnaire  provides  a  fast,  inexpensive,  and  convenient  means  for 
gathering  information  of  interest  to  a  researcher.  The  major  criticism 
of  the  questionnaire  is  that  it  depends  too  heavily  on  written  com¬ 
munication.  The  respondent  has  no  means  of  getting  feedback  whenever  he 
perceives  a  question  as  being  either  unclear  or  ambiguous. 

Personal  interviews  lend  themselves  to  original  thought  more  readily 
than  the  structured  format  of  the  questionnaire.  Furthermore,  feedback  is 
an  integral  part  of  the  interview  process.  One  major  criticism  of  the 
personal  interview  is  that  it  is  very  time  consuming  and  therefore  can 
only  be  used  to  reach  a  limited  population.  Another  criticism  is  that  it 
is  too  easy  for  the  interviewer  to  inject  his  own  beliefs,  attitudes  or 
biases  into  the  interview  process. 

A  series  of  nine  interviews  were  conducted  with  selected  cost  analysts 


II.  Methodology 


Introduction  and  Approach 

The  information  requested  by  the  research  questions  at  the  end  of 
chapter  one  lends  itself  most  appropriately  to  the  interrogation  process, 
or  more  commonly,  the  survey  instrument.  A  survey  is  one  of  the  most 
popular  means  of  gathering  data.  Its  popularity  is  directly  attributable 
to  its  versatility,  which  is  the  greatest  strength  of  a  survey  instrument. 
However,  one  of  the  major  criticisms  of  the  survey  is  that  the  respondent 
can  intentionally  give  false  or  misleading  answers.  Another  key  criticism 
is  the  belief  by  most  people  that  designing  an  interview  or  questionnaire 
is  a  simple  task.  This  is  hardly  the  case  and  is  most  likely  the  reason 
for  much  of  the  heavy  criticism  of  survey  results.  (Emory,  1980:213-214). 

Emory  identifies  two  different  modes  for  soliciting  information.  He 
identifies  the  personal  mode  as  a  "relationship  between  interviewer  and 
interviewee,"  and  the  impersonal  mode  as  "two-way  communication"  via  a 
printed  instrument  (i.e.,  questionnaire).  He  also  describes  a  combination 
of  the  two  communication  modes  as  a  "mixed  mode"  (Emory,  1980:214-215). 

Determining  the  appropriate  communication  mode  for  designing  the  sur¬ 
vey  instrument  was  the  first  task  to  be  accomplished.  Careful  con¬ 
sideration  of  the  information  requirements  led  to  adoption  of  both 
personal  and  impersonal  modes  of  communicating  with  the  respondents.  When 
attributes  or  opinions  are  the  item  of  interest  to  a  researcher  and  the 
target  population  is  fairly  large  and  geographically  widespread,  the 
questionnaire  is  most  often  the  only  practical  means  of  gathering  needed 
data.  Although  alternative  approaches  may  be  possible,  the  expense  and 
difficulty  involved  in  pursuing  these  alternates  can  far  outweigh  the 
potential  benefits  of  the  information  acquired  (Emory,  1980:213-214).  The 
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ments  are  also  important  to  this  research  effort.  In  addition,  the 
opinions  of  experienced  analysts  are  solicited  to  ascertain  the  usefulness 
of  AFIT  PCE  courses  to  the  job  performance  of  Air  Force  cost  analysts. 
Finally,  suggestions  by  experienced  analysts  for  improving  the  educational 
competence  of  all  Air  Force  cost  analysts  will  be  gathered  and  evaluated. 
Given  these  objectives  the  following  research  questions  were  explored: 

1.  What  educational  requirements  do  experienced  Air  Force  cost  analysts 
identify  as  being  most  useful  to  doing  their  job? 

2.  What  methods  of  education  do  experienced  Air  Force  cost  analysts 
suggest  for  teaching  requirements  they  identify  as  being  most  useful  to 
doing  their  jobs? 

3.  How  do  experienced  Air  Force  cost  analysts  rate  cost  analysis  related 
courses  in  the  AFIT  PCE  program  in  terms  of  their  usefulness  to  job  per¬ 
formance? 

4.  According  to  experienced  Air  Force  cost  analysts,  how  can  the  educa¬ 
tional  competence  (adequate  knowledge  for  job  performance)  be  improved  for 
all  Air  Force  cost  analysts? 
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will  help  to  strengthen  the  professionalism  of  the  estimating  community 
(Dollars  and  Sense,  1984). 

Another  prime  contribution  to  the  professional  development  of  the 
field  of  cost  analysis  is  the  AFIT  cost  analysis  Education  With  Industry 
(EWI)  program.  This  program  was  the  result  of  an  inquiry  by  General  Slay, 
a  formal  commander  of  Air  Force  Systems  Command  (AFSC).  The  AFSC  com¬ 
mander  solicited  the  Boeing  Company's  interest  in  a  program  to  expose 
senior  field  grade  officers  to  management  techniques  used  in  the  commer¬ 
cial  environment.  The  present  EWI  program  allows  selected  Air  Force  offi¬ 
cers  in  the  cost  analysis  specialty  to  be  assigned  to  a  contractors  plant 
to  experience  first-hand  the  business  practices  of  the  commercial  sector 
with  hopes  that  they  will  improve  the  management  practices  of  defense 
weapon  systems  acquisition  (Temple,  1981:5). 

There  is  also  a  master's  degree  program  offered  by  AFIT.  This 
program,  which  began  in  June  of  1982,  awards  its  graduates  a  master's 
degree  in  systems  management  with  a  concentration  in  cost  analysis  (Goven, 
1983:11).  This  program  also  grants  its  military  graduates  an  advanced 
degree  identifier  (i.e.,  1ASA)  which  further  documents  the  development  of 
the  cost  analysis  profession  in  addition  to  its  primary  purpose  of 
controlling  job  assignments  ("1ASA"  is  merely  a  code  identifying  an  aca¬ 
demic  specialty  in  cost  analysis,  it  is  not  an  acronym  with  a  direct 
meaning) . 


Research  Objectives/Questions 

The  objective  of  this  research  is  to  determine  the  educational 
requirements  that  experienced  (at  least  four  years  in  the  cost  analysis 
field)  Air  Force  cost  analysts  identifiy  as  being  most  useful  to  job  per¬ 


formance.  The  various  methods  for  acquiring  these  educational  require- 
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providing  encouragement;  for  continued  professional  education  and  starting 
a  formal  certification  process,  the  PO  program  has  become  a  milestone  in 
the  professional  development  of  the  cost  and  price  analysis  fields 
(Kankey,  1982:27,39).  The  certification  process  requires  the  completion 
of  a  total  of  eight  courses  in  cost  and  price  analysis.  Four  of  the  cour¬ 
ses  are  core  requirements  and  the  other  four  are  chosen  from  a  list  of 
several  electives  (Novak,  1982:41). 

Many  on-the-job  training  programs  depend  heavily  on  the  Professional 
Continuing  Education  program  offered  by  AFIT  to  supplement  the  training 
of  new  cost  analysts.  The  Professional  Designation  in  Cost  Analysis 
program  has  its  roots  in  the  PCE  program.  Establishing  a  standard  educa¬ 
tion  plan  for  supplementing  OJT  programs  may  be  a  viable  step  towards 
standardizing  the  educational  competence  of  cost  analysts  Air  Force  wide. 

The  Institute  of  Cost  Analysis  is  also  actively  involved  in  a 
Certified  Cost  Analysis  program.  This  program,  which  is  designed  to 
enhance  the  professional  stature  of  cost  and  price  analysts  throughout 
the  financial  community,  is  by  far  the  most  essential  function  of  the 
institute.  The  certification  of  both  government  and  private  sector  ana¬ 
lysts  makes  the  certification  process  even  more  important  to  the  more  than 
250  cost  and  price  analysts  who  have  received  certificates  acknowledging 
their  many  contributions  to  the  professional  development  of  the  field  of 
cost  analysis  (Goven,  1983:10).  The  National  Estimating  Society  is 
another  professional  cost  analysis  organization  which  also  offers  a  cer¬ 
tification  program.  This  program  has  awarded  some  642  Certified 
Professional  Estimator  certificates  to  both  private  sector  and  government 
estimators.  As  of  August  4,  1984  this  program  gives  a  written  test  to 
evaluate  potential  certificate  recipients.  This  new  objective  evaluation 
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*  GS-501,  Accounting  Clerical 

*  GS-1515,  Operations  Research  Analyst  (Goetsch,  1980:11) 


*  GS-343,  Management  Analyst 

*  GS-345 ,  Program  Analyst 

*  GS-560,  Budget  Analyst  (Temple,  1981:8) 

It  is  quite  evident  that  this  lack  of  a  common  civilian  cost  analyst  job 
series  in  the  Air  Force  only  complicates  the  issue  of  identifying  educa¬ 
tional  requirements.  Clarence  Goetsch  expressed  his  concern  on  the  job 
series  issue,  but  hastened  to  add  that  there  were  some  pros  and  cons  to 
be  considered: 

PROS:  *  It  would  insure  that  only  qualified  and  experienced  cost 
analysts  could  compete  for  cost  jobs. 

*  The  cost  analysis  profession  would  not  be  officially 
recognized  until  job  qualifications,  education  and 
experience  requirements,  and  typical  duties  are  standard¬ 
ized. 

CONS:  *  It  would  deemphas ize  the  multidiscipl inary  (financial, 

mathematical,  science,  and  engineering)  background  needed 
for  cost  analysis. 

*  The  installation  of  a  specialized  series  and  standards  for 
cost  analysts  limits  the  opportunity  for  intake  and  utiliza¬ 
tion  of  all  other  "multidiscipline"  professional  people 
(Goetsch,  1980:11). 

It  seems  that  until  a  common  ground  is  established  in  the  specification 
of  job  series  and  educational  qualifications,  the  professionalism  of  all 
Air  Force  cost  analysts  will  remain  questionable. 

In  light  of  the  concern  for  professionalism  and  standardization  of 
education  the  establishment  of  a  Professional  Designation  (PD)  program  in 
cost  and  price  analysis  has  enhanced  professional  development  within  the 
field  of  cost  analysis.  This  designation,  which  is  offered  by  the  Air 
Force  Institute  of  Technology  in  conjunction  with  the  Institute  of  Cost 
Analysis,  was  first  implemented  in  August  of  1981  (Kankey,  1982:27).  By 
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be  found  in  Air  Force  Regulation  36-1,  Officer  Air  Force  Speciality.  The 
regulation  identifies  undergraduate  specialization  in  business  administra¬ 
tion,  economics,  mathematics,  engineering,  statistics,  or  operations 
research  with  a  minimum  of  tweLve  hours  in  accounting  as  "desirable" 

(AFR  36-1).  There  is  no  mention  of  whether  the  twelve  hours  of  accounting 
should  be  quarter  hours  or  semester  hours.  Also  the  use  of  the  word 
"desirable"  leads  one  to  believe  that  there  really  are  no  stringent  edu¬ 
cational  considerations  other  than  the  undergraduate  degree  required  of 
all  Air  Force  officers.  In  February  1984  a  telephone  interview  with 
Captain  Bill  Pazeretsky,  an  Air  Force  personnel  advisor  for  military  cost 
analysts  at  the  Air  Force  Military  Personnel  Center  (AFMPC),  confirmed 
beliefs  in  this  area  (Pazeretsky).  The  Air  Force  chooses  its  military 
costs  analysts  based  upon  a  subjective  evaluation  of  the  records  of 
potential  analysts.  This  evaluation  is  performed  by  the  Palace  Dollar 
team,  a  group  of  AFMPC  officers  consisting  of  three  military  personnel 
advisors  in  the  cost  analysis  field.  Although  mathematical  and  technical 
competence  are  key  qualifications  deemed  desirable,  there  is  no  require¬ 
ment  for  an  academic  background  which  satisfies  these  qualifications 
(Pazeretsky) . 

Air  Force  civilian  cost  analysts  lack  what  little  educational  direc¬ 
tion  is  provided  to  their  military  colleagues.  Investigation  revealed 
that  there  is  no  equivalent  civil  service  regulation  to  Air  Force 
regulation  36-1  which  identifies  the  educational  qualifications  of  civi¬ 
lian  cost  analysts.  In  fact  there  is  no  unique  job  series  identification 
specifically  for  the  civilian  cost  analyst.  The  Air  Force  places  its 
civilian  cost  analysts  in  either  of  several  job  series: 


*  GS-301,  Admin  and  Managerial  Analyst 


Although  the  papers  relate  the  Army  perspective  on  the  issue,  the  direct 
application  of  this  information  to  the  sister  services  cannot  be 
neglected.  Everything  from  the  history  and  definition  of  cost  analysis  to 
Che  first  drafts  of  professional  and  qualification  standards  for  cost  ana¬ 
lysts  are  included  in  this  collection  of  papers  (Allen,  1980).  For 
anyone  interested  in  the  professional  development  of  the  cost  analysis 
field  this  collection  of  papers  is  the  most  exhaustive  compilation  of  cost 
analysis  information  encountered  in  all  the  literature  reviewed  by  this 
research  effort. 

Careful  review  of  existing  literature  which  included  a  Defense 
Technical  Information  Center  search  (3  January  1984)  and  a  Federal  Legal 
Information  through  Electronics  search  (23  January  1984)  yielded  only  a 
limited  number  of  articles  specifically  addressing  the  educational  quali¬ 
fications  of  either  commercial  or  Air  Force  cost  analysts. 

In  one  of  the  few  articles  discovered,  Clarence  H.  Goetsch,  a  leading 
defense  cost  analyst,  stated,  "To  my  knowledge,  no  one  has  published  an 
article  in  any  magazine  (military  or  civilian)  on  qualifications  of  a  cost 
analyst  (Goetsch,  1980:10)."  Goetsch  goes  on  to  discuss  the  lack  of 
information  on  cost  analyst  qualifications  and  how  he  believes  the  mili¬ 
tary  cost  analyst  can  be  improved  as  a  more  valuable  resource.  He  also 
mentions  how  his  research  of  the  business  and  scientific  periodical 
indexes  (from  1969  to  1979)  and  defense  files  revealed  only  one  article  on 
a  cost  analyst  and  his  qualifications.  This  article,  written  by  Admiral 
Rickover,  raised  many  key  questions  on  the  qualifications  of  a  cost  ana¬ 
lyst,  but  failed  to  provide  any  of  the  answers  (Goetsch,  1980:10). 

There  is  only  a  vague  list  of  the  educational  qualifications  for  mili¬ 
tary  officers  entering  the  cost  analysis  specialty  (674X).  This  list  can 
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Sc iences  (SPSS)  provided  an  efficient  means  for  generating  the  necessary 
tables.  The  data  was  used  to  generate  thirty-five  contingency  tables  com¬ 
paring  the  frequencies  of  ratings  (of  one  to  five)  to  educational  methods 
(which  consisted  of  fifteen  possible  outcomes),  controlling  for  subject. 
This  provided  a  possible  seventy-five  blocks  in  each  of  the  thirty-five 
tables,  however,  most  of  the  tables  contained  less  than  seventy-five 
blocks  since  the  program  only  printed  those  rows  and  columns  which 
actually  contained  data  inputs  on  a  given  subject.  These  tables  are 
included  in  Appendix  C.  A  detailed  discussion  of  the  Crosstabs  procedure 
and  its  general  uses  is  presented  in  Statistical  Package  for  the  Social 
Sciences  Second  Edition  (Nie  and  others,  1975:218-245). 

The  first  research  objective  is  to  identify  those  subjects  which 
should  be  included  in  a  cost  analysis  educational  program.  This  was 
accomplished  by  utilizing  a  decision  rule  which  resulted  from  a  careful 
consideration  of  possible  distributions  of  responses  across  the  usefulness 
rating.  The  median  rating  in  conjunction  with  a  restriction  on  the 
distribution  of  responses,  is  used  to  make  a  recommendation  as  to  the 
value  of  a  given  subject.  There  are  several  possible  recommendations. 

Gore.  A  core  subject  should  be  included  as  a  mandatory  requirement  in 
a  cost  analysis  educational  program.  A  subject  is  identified  as  "core" 
when  respondents  universally  indicated  that  the  subject  was  very  valuable. 
The  recommendation  that  a  subject  be  included  in  the  core  will  be  made 
whenever  one  of  the  following  criteria  is  met: 

1.  At  least  50  percent  of  the  respondents  felt  the  subject  was 

extremely  valuable,  while  less  than  20  percent  of  the  respond¬ 
ents  expressed  an  opposing  view. 

Specific  Criteria:  Median  category  is  1  or  2. 

Sum  of  responses  in  categories  4  and  5 
combined  is  less  than  20  percent. 
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2.  Less  then  50  percent  but  more  than  30  percent  felt  the  subject 
was  extremely  valuable,  and  80  percent  or  more  of  the  respondents 
felt  the  subject  was  at  least  valuable. 

Specific  Criteria:  Median  category  is  3. 

Sum  of  responses  in  categories  1  and  2  com¬ 
bined  is  at  least  30  percent. 

Sum  of  responses  in  categories  1,  2  and  3 
combined  is  at  least  70  percent. 

In  both  situations  the  core  recommendation  is  justified  by  the  fact  that 
the  preponderance  of  the  respondents  identified  the  course  as  very 
valuable,  and  there  was  no  significant  countervailing  opinion. 

Elective.  An  elective  should  be  included  in  a  cost  analysis  educa¬ 
tional  program,  but  taken  only  when  it  is  relevant  to  the  analyst's  par¬ 
ticular  job.  This  determination  can  best  be  made  by  the  individual  ana¬ 
lyst  and  his  supervisor.  A  subject  is  identified  as  an  "elective"  either 
when  the  respondents  express  a  strong  bipolar  opinion  concerning  the  value 
of  a  given  subject,  or  when  there  are  no  strong  opinions  expressed  at 
either  end  of  the  spectrum.  The  recommendation  that  a  subject  be  included 
as  an  elective  will  be  made  whenever  one  of  the  following  criteria  is  met: 

3.  At  least  50  percent  of  the  respondents  felt  the  subject  was  at 
least  valuable,  yet  at  least  30  percent  felt  that  the  subject  was 
of  little  or  no  value.  This  represents  the  case  of  the  strong 
bipolar  opinions. 

Specific  Criteria:  Median  category  is  3. 

Sum  of  responses  in  categories  4  and  5  is  at 
least  thirty  percent. 

4.  While  les3  than  50  percent  of  the  respondents  felt  the  subject  was 
of  value,  at  least  50  percent  of  the  respondents  felt  it  was  at 
least  valuable.  Also  less  than  30  percent  of  the  respondents  felt 
the  subject  was  extremely  valuable  and  less  than  30  percent  of  the 
respondents  felt  the  subject  was  of  little  or  no  value.  This 
represents  the  case  of  no  significant  extreme  opinions. 

Specific  Criteria:  Median  category  is  3,  but  the  total  responses 

in  this  category  are  less  than  50  percent. 

Sum  of  the  responses  in  categories  4  and  5 
combined  is  less  than  30  percent. 

Sum  of  the  responses  in  categories  1  and  2 
combined  is  also  less  than  30  percent. 
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The  elective  recommendation  is  justified  by  the  fact  that  in  both 
situations  there  is  no  strong  opinion  overwhelmingly  in  favor  of 
establishing  the  subject  as  either  very  valuable,  or  of  little  or  no 
value. 

Excluded.  An  excluded  subject  should  not  be  offered  as  part  of  a 
cost  analysis  educational  program.  The  subject  is  not  relevant  to  either 
the  general  nor  the  specific  knowledge  requirements  of  a  cost  analyst.  A 
subject  is  identified  as  "excluded"  when  the  preponderance  of  respondents 
indicate  that  it  is  of  little  or  no  value.  The  recommendation  that  a  sub¬ 
ject  be  excluded  will  be  made  whenever  the  following  criteria  is  met: 

5.  At  least  50  percent  of  the  respondents  identify  the  course  as  of 
little  or  no  value,  and  less  than  20  percent  feel  that  it  is  very 
valuable. 

Specific  Criteria:  Median  category  is  4  or  5. 

Sum  of  the  responses  in  categories  1  and  2 
combined  is  less  than  20  percent. 

The  distribution  of  opinions  expressed  by  the  respondents  may  not  always 
be  compatible  with  a  single,  unambiguous  recommendation.  Under  these  cir¬ 
cumstances,  a  subject  will  receive  a  dual  recommendation,  with  the  primary 
recommendation  stated  first.  An  acceptable  cost  analysis  educational 
program  can  be  developed  by  following  the  primary  recommendations. 

However,  if  resource  cons  taints  mandate  that  the  program  be  reduced  in 
scope,  the  core/elective  subjects  should  first  be  downgraded  to  an  elec¬ 
tive  status.  If  a  further  reduction  is  required,  the  elective/excluded 
subjects  should  then  be  eliminated.  Any  further  reduction  would  result  in 
an  inadequate  educational  program.  In  the  event  that  additional  resources 
are  available,  primary  priority  should  be  given  to  expanding  the  program 
by  adding  the  excluded/elective  subjects.  Secondary  priority  should  be 
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given  Co  upgrading  Che  elecCive/core  subjeccs  Co  a  mandatory  sCaCus.  Any 
addicional  accempc  Co  expand  or  upgrade  Che  program  would  represenc  an 
inefficienc  use  of  resourses.  The  four  possible  dual  recommendaCions  and 
Cheir  criceria  are: 

Core/elective.  A  subjecC  receives  chis  recommendacion  whenever  Chere  is 
significanc  opposicion  co  Che  majoricy  opinion  chac  Che  subjecC  is  excremely 
valuable,  or  when  Che  majoricy  feels  ChaC  Che  subjecC  is  valuable,  and  Che 
opinions  aC  eicher  exCreme  are  insignif icant.  A  "core/elecCive"  recommen¬ 
dacion  is  made  when  a  subjecC  meeCs  che  following  criceria. 

6.  AC  lease  50  percenC  of  Che  respondencs  idencify  Che  subjecC 

as  excremely  valuable  while  a  significanc  number  of  respondencs 
indicaCe  Che  subjecC  as  being  of  liCCle  or  no  value. 

Specific  Criceria:  Median  caCegory  is  1  or  2. 

Sum  of  Che  responses  in  caCegories  4  and  5 
combined  is  aC  lease  20  percenC  buC  less  Chan 
30  percenC. 

7.  AC  lease  50  percenC  of  respondencs  idencify  Che  subjecC  as  being 
of  value,  and  neicher  extreme  posicion  received  a  significanc 
represencacion. 

Specific  Criceria:  The  Cocal  responses  in  caCegory  3  exceed  50 

percenC,  making  it  Che  median  category. 

Sum  of  Che  responses  in  categories  1  and  2  is 
less  Chan  30  percent  and  the  sum  or  responses 
in  categories  4  and  5  is  less  than  30  percent. 

E lective/Core.  A  subject  receives  this  recommendation  whenever  there  is 
substantial  opposition  to  the  majority  opinion  that  the  subject  is  extre¬ 
mely  valuable.  An  "elective/core"  recommendacion  is  made  when  the 
following  criteria  is  met. 

8.  AC  least  50  percent  of  the  respondencs  idencify  Che  subject  as 
extremely  valuable  while  a  substantial  number  of  the  respondents 
indicaCe  thaC  the  subjecC  is  of  little  or  no  value. 

Specific  Criteria:  Median  category  is  I  or  2. 

Sum  of  the  responses  in  categories  4  and  5 
combined  is  at  least  30  percent. 

E lect ive/ exc luded.  A  subject  receives  this  recommendacion  whenever  there 


is  a  substantial  opposition  to  the  majority  opinion  that  the  subject  is  of 
little  or  no  value.  An  "elective/excluded"  recommendation  is  made  when 


the  following  criteria  is  met. 

9.  At  least  50  percent  of  respondents  consider  a  subject  to  be  of 
little  or  no  value  while  a  substantial  number  of  the  respondents 
indicate  that  the  subject  is  extremely  valuable. 

Specific  Criteria:  Median  category  is  4  or  5. 

Sum  of  the  responses  in  categories  1  and  2 
combined  is  at  least  30  percent. 

Excluded/elective.  A  subject  recieves  this  recommendation  whenever  there 
is  a  significant  opposition  to  the  majority  opinion  that  the  subject  is  of 
little  or  no  value.  An  "excluded/elective"  recommendation  is  made  when 
the  following  criteria  is  met. 

10.  At  least  50  percent  of  respondents  consider  the  subject  to  be  of 
little  or  no  value  while  a  significant  number  of  the  respondents 
indicate  that  the  subject  is  extremely  valuable. 

Specific  Criteria:  Median  category  is  4  or  5. 

Sum  of  the  responses  in  categories  1  and  2 
combined  is  at  least  20  percent  but  less  than 
30  percent. 

The  list  below  summarizes  the  criteria  used  to  identify  the  content  of  the 
recommended  cost  analysis  educational  program.  The  criteria  incorporates 
the  median  rating  and  restrictions  on  the  distribution  of  responses  to 
establish  the  appropriate  recommendation.  The  following  decision  criteria 


and  recommendations 

apply: 

Recommendat ion 

Criteria 

Number 

Median 

Categories 

Characteritics  of 
the  Distribution 

Core 

1 

1  or  2 

Categories  4  and  5  *  less  than 
20  percent. 

2 

3 

Categories  1  and  2  »  at  least 
30  percent. 

Categories  1,  2  and  3  *  at 
least  70  percent. 
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Elective 


3 


Excluded 


Core/elective 


Elective/core  8 

Elective/excluded  9 

Excluded/elective  10 


3  Categories  4  and  5  *  at  least 

30  percent. 

3  Categories  1  and  2  *  less  than 

30  percent. 

Categories  4  and  5  *  less  than 
30  percent. 

Category  3  3  less  than  50 
percent. 

4  or  5  Categories  1  and  2  3  less  than 

20  percent. 

1  or  2  Categories  1  and  2  3  at  least 

20  percent  but  less  than  30 
percent. 

3  Categories  1  and  2  3  less  than 

30  percent. 

Categories  4  and  5  3  less  than 
30  percent. 

Category  3  3  at  least  50 
percent 

1  or  2  Categories  4  and  5  3  at  least 

30  percent. 

4  or  5  Categories  1  and  2  3  at  least 

30  percent. 

4  or  3  Categories  1  and  2  3  at  least 

20  percent  but  less  than  30 
percent. 


A  second  set  of  criteria,  based  primarily  on  the  mode,  was  established 
to  identify  the  appropriate  educational  method  for  teaching  a  particular 
subject.  The  mode  is  the  educational  method  which  received  the  greatest 
number  of  responses.  However,  it  is  possible  for  a  distribution  to  have 
more  chan  one  mode.  Furthermore,  it  is  possible  for  some  educational 
methods  to  receive  a  substantial  number  of  responses  and  yet  not  qualify 
as  a  mode.  To  account  for  this  phenomenon,  three  separate  criteria  were 


developed  for  identifying  the  preferred  educational  methods. 

1.  The  educational  method  recommended  by  the  highest  percentage  of 
respondents  (mode)  is  identified  as  most  preferred. 

2.  Educational  methods  recommended  by  a  significant  segment  of  the 
population  (at  least  30  percent)  are  also  identified  as  preferred. 
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3.  If  Che  number  of  respondents  recommending  a  particular  educational 
method  equals  or  exceeds  80  percent  of  the  number  of  respondents 
favoring  the  mode,  this  method  is  also  identified  as  preferred. 

Respondents  were  permitted  to  recommend  more  than  one  educational  method 

for  any  given  subject.  Therefore,  the  number  of  responses  may  exceed  the 

number  of  respondents.  Percentages  reflect  the  proportion  of  respondents 

recommending  a  particular  educational  method  for  a  given  subject. 

Consequently,  these  percentages  may  total  to  more  than  100  percent. 

If  the  recommendation  was  made  that  a  subject  be  included  in  the 
"core",  the  opinion  of  all  of  the  respondents  was  taken  into  account  when 
identifying  the  preferred  educational  method  since  every  cost  analyst 
would  be  required  to  develop  an  acceptable  level  of  competence  in  the 
"core"  subject.  However,  if  the  recommendation  was  made  that  a  subject 
be  included  as  an  "elective",  only  the  opinions  of  those  respondents  who 
felt  the  subject  was  at  least  valuable  were  taken  into  account  when  iden¬ 
tifying  the  preferred  educational  method.  This  was  done  in  the  expec¬ 
tation  that  only  those  respondents  would  attempt  to  develop  competence  in 
these  subjects.  Those  respondents  who  indicated  that  the  subject  was  of 
little  or  no  use  would  probably  forego  the  opportunity  to  take  these  sub¬ 
jects.  Therefore,  their  opinions  on  the  preferred  educational  method  were 
ignored. 

The  final  question  addressing  the  issue  of  how  to  improve  the  educa¬ 
tional  competence  of  Air  Force  cost  analysts  was  designed  to  provide  each 
respondent  with  the  opportunity  to  present  his  views  on  subjects  not 
addressed  in  the  structured  portion  of  the  questionnaire.  These  responses 
are  recorded  in  Appendix  B.  The  analysis  of  the  questionnaire  will 
conclude  with  a  summary  and  analysis  of  these  responses. 
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III.  Analys is  Results 


Responses 

Questionnaires  were  distributed  to  the  total  CONUS  population  of  386 
Air  Force  cost  analysts.  However,  only  239,  or  62  percent,  of  the 
questionnaires  distributed  were  returned.  Seventy-five  of  the  question¬ 
naires  returned  were  omitted  from  the  analysis  because  the  respondents 
did  not  meet  the  minimum  requirement  of  at  least  four  years  experience  in 
the  cost  analysis  field.  Another  thirteen  questionnaires  were  omitted 
because  they  were  either  returned  blank  or  were  not  properly  filled  out. 
This  left  the  number  of  useable  questionnaires  at  151.  This  amount  was 
less  than  half  (39  percent)  of  the  number  distributed,  but  it  provided  a 
reasonable  sample  for  the  subsequent  analysis.  Useable  responses  ranged 
across  several  commands  with  the  bulk  of  respondents  being  from  Air  Force 
Systems  Command  (AFSC).  This  was  expected  since  most  of  Air  Force  Cost 
Analysts  are  in  AFSC. 

Subject  by  Subject  Analysis  Results 

The  following  analysis  identifies  each  subject  in  the  questionnaire 
and  presents  the  relevant  statistics  on  each.  The  criteria  developed 
in  chapter  two  are  applied  and  appropriate  recommendations  are  made 
concerning  the  content  of  the  cost  analysis  educational  program  and  the 
preferred  educational  methods.  The  number  in  paranthesis  preceeding  each 
subject  represents  the  order  in  which  the  subject  appears  in  the  question¬ 
naire.  A  description  of  the  subject  is  provided  in  the  questionnaire, 
located  in  appendix  A.  A  detailed  analysis  of  each  subject  follows: 


(1)  Financial  and  Cost  Accounting.  The  distribution  of  responses  to 
the  usefulness  rating  is: 


PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

19.7 

19.7 

100.1 

2 

Extremely  Useful 

35.4 

55.1 

80.4 

3 

Useful 

36.1 

91.2 

45.0 

4 

Of  Little  Use 

8.2 

99.4 

8.9 

5 

Useless 

0.7 

100.1 

0.7 

Since  the 

median  category  is 

2,  and  the 

total  percent  of 

responses  in 

categories  4  and  5  combined  is  8.9  percent,  this  distribution  satisfies 
criteria  number  1.  Therefore  the  recommendation  is  that  this  subject  be 
included  in  the  core. 

The  distribution  of  responses  for  the  educational  method  for  this 
recommendation  is: 


EDUCATIONAL 

PERCENT  OF  RESPONDENTS 

METHOD 

CORE 

OJT 

17.7 

PCE 

19.7 

UG 

75.5 

GRAD 

8.2 

An  undergraduate  education  is  identified  as  the  most  preferred  method  for 
acquiring  competence  in  this  subject.  No  other  educational  method  met 
remaining  criteria. 


(2)  Managerial  Accounting.  The  distribution  of  responses  to  the  use¬ 
fulness  rating  is: 


PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

25.0 

25.0 

100.0 

2 

Extremely  Useful 

38.5 

63.5 

75.0 

3 

Useful 

28.4 

91.9 

36.5 

4 

Of  Little  U3e 

7.4 

99.3 

8.1 

5 

Useless 

0.7 

100.0 

0.7 
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Since  Che  median  category  is  2,  and  Che  total  percent  of  responses  in 
categories  4  and  5  combined  is  8.1  percent,  this  distribution  satisfies 
criteria  number  1.  Therefore,  the  recommendation  is  that  this  subject  be 
included  in  the  core. 

The  distribution  of  responses  for  the  educational  method  for  this 
recommendation  is: 


EDUCATIONAL 

METHOD 


PERCENT  OF  RESPONDENTS 
CORE 


OJT 

PCE 

UG 

GRAD 


14.2 

19.6 

61.5 

23.0 


An  undergraduate  education  is  identified  as  the  most  preferred  method  for 


acquir ing 

competence  in  this 

subject.  No 

other  educational 

method  met  the 

remaining 

criteria. 

(3)  Microeconomics.  The 

:  distribution 

of  responses  to 

the  usefulness 

rating  is: 

PERCENT 

CUMULATIVE  PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

6.3 

6.3 

100.1 

2 

Extremely  Useful 

15.3 

21.6 

93.8 

3 

Useful 

52.1 

73.7 

78.5 

4 

Of  Little  Use 

24.3 

98.0 

26.4 

5 

Useless 

2.1 

100.1 

2.1 

Since  the  total  percent  of  responses  in  category  3  is  52.1  percent,  the 
total  percent  of  responses  in  categories  1  and  2  combined  is  21.6  percent, 
and  the  total  percent  of  responses  in  categories  4  and  5  combined  is  26.4 
percent,  this  distribution  satisfies  criteria  number  7.  The  primary  recom¬ 
mendation  is  that  this  subject  be  included  in  the  core,  and  the  secondary 
recommendation  i3  that  it  be  taught  as  an  elective. 

The  distribution  of  responses  for  the  educational  method  for  each  of 
the  two  recommendations  is: 
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EDUCATIONAL 

PERCENT  OF 

RESPONDENTS 

METHOD 

CORE 

ELECTIVE 

OJT 

9.7 

9.4 

PCE 

10.4 

12.3 

UG 

77.8 

75.5 

GRAD 

13.2 

16.0 

An  undergraduate  education  is  identified  as  the  most  preferred  method  for 


acquiring  competence  in  this  subject  regardless  of  whether  the  subject  was 
included  in  the  educational  program  as  a  core  requirement  or  as  an  elec¬ 
tive.  No  other  educational  method  met  either  of  the  two  remaining  criteria. 
(4)  Macroeconomics.  The  distribution  of  responses  to  the  usefulness 


rating  is: 

PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

13.7 

13.7 

100.1 

2 

Extremely  Useful 

27.4 

41.1 

86.4 

3 

Useful 

42.5 

83.6 

59.0 

4 

Of  Little  Use 

15.1 

98.7 

16.5 

5 

Useless 

1.4 

100. 1 

1.4 

Since  the  median  category  is  3,  the  total  percent  of  responses  in  cate¬ 
gories  1  and  2  combined  is  41.1  percent,  and  the  total  percent  of  respon¬ 
ses  in  categories  1,  2  and  3  combined  is  83.6  percent,  this  distribution 
satisfies  criteria  number  2.  Therefore,  the  recommendation  is  that  this 
subject  be  included  in  the  core. 

The  distribution  of  responses  for  the  educational  method  for  this 
recommendation  is.: 


EDUCATIONAL 

METHOD 


PERCENT  OF  RESPONDENTS 
CORE 


OJT 

PCE 

UG 

GRAD 


11.0 

16.4 

66.4 
22.6 


An  undergraduate  education  is  identified  as  the  most  preferred  method  for 
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acquiring  competence  in  this  subject.  No  other  educational  method  met 
either  of  the  two  remaining  criteria. 


(5) 

Technological  Forecasting.  The 

distribution  of 

responses  to  the 

usefulness  rating  is: 

PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

19.2 

19.2 

100.0 

2 

Extremely  Useful 

43.8 

63.0 

80.8 

3 

Useful 

31.5 

94.5 

37.0 

4 

Of  Little  Use 

4.1 

98.6 

5.5 

5 

Useless 

1.4 

100.0 

1.4 

Since  the  median  category  is  2,  and  the  distribution  of  responses  in  cate¬ 
gories  4  and  5  combined  is  5.5  percent,  this  distribution  satisfies  cri¬ 
teria  number  1.  Therefore,  the  recommendation  is  that  this  subject  be 
included  in  the  core. 

The  distribution  of  responses  for  the  educational  method  for  this 
recommendation  is: 


EDUCATIONAL 

PERCENT  OF  RESPONDENTS 

METHOD 

CORE 

OJT 

19.2 

PCE 

52.1 

UG 

24.7 

GRAD 

28.8 

PCE  is  identified  as  the  most  preferred  method  for  acquiring  competence 
in  this  subject.  No  other  educational  method  met  the  remaining  criteria. 

(6)  Federal  Financial  Management.  The  distribution  of  responses  to 
the  usefulness  rating  is: 

PERCENT  CUMULATIVE  PERCENT  RESPONSE 


CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

30.1 

30.1 

99.9 

2 

Extremely  Useful 

34.9 

65.0 

69.8 

3 

Use  fu  1 

30.8 

95.8 

34.9 

4 

Of  Little  Use 

3.4 

99.2 

4.1 

5 

Useless 

0.7 

99.9 

0.7 

Since  the  median  category  is  2,  and  the  total  percent  of  responses  in 
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categories  4  and  5  combined  is  4.1  percent,  this  distribution  satisfies 
criteria  number  1.  Therefore,  the  recommendation  is  that  this  subject  be 
included  in  the  core. 

The  distribution  of  responses  for  the  educational  method  for  this 
recommendation  is: 


EDUCATIONAL 

METHOD 


PERCENT  OF  RESPONDENTS 
CORE 


QJT 

PCE 

UG 

GRAD 


32.9 

70.5 

6.2 

6.2 


PCE  is  identified  as  the  most  preferred  method  for  acquiring  competence 
in  this  subject.  The  criteria  also  identify  OJT  as  an  appropriate  educa¬ 
tional  method. 

(7)  Managerial  Finance.  The  distribution  of  responses  to  the  use¬ 
fulness  rating  is: 

PERCENT  CUMULATIVE  PERCENT  RESPONSE 


CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

4.8 

4.8 

100.0 

2 

Extremely  Useful 

22.1 

26.9 

95.2 

3 

Useful 

44.8 

71.7 

73.1 

4 

Of  Little  Use 

26.2 

97.9 

28.3 

5 

Useless 

2.  1 

100.0 

2.1 

Since  the  median  category  is  3,  the  total  percent  of  responses  in  category 

3  is  44.8  percent,  the  total  percent  of  responses  in  categories  1  and  2 
combined  is  26.9  percent,  and  the  tital  percent  of  responses  in  categories 

4  and  5  combined  is  28.3  percent,  this  distribution  satisfies  criteria 
number  4.  Therefore,  the  recommendation  is  that  this  subject  be  taught  as 
an  elective. 

An  undergraduate  education  is  identified  as  the  most  preferred  method 
for  acquiring  competence  in  this  subject.  No  other  educational  method  met 
the  remaining  criteria. 
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The  distribution  of  responses  for  the  educational  method  is: 


EDUCATIONAL 

METHOD 


PERCENT  OF  RESPONDENTS 
ELECTIVE 


OJT 

PCE 

UG 

GRAD 


14.4 

20.1 

58.7 

25.0 


(8) 

Management  and  Behavior  in  Organ 

izations.  The 

distribution  of 

responses 

to  the  usefulness 

rating  is: 

PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

6.3 

6.3 

100.1 

2 

Extremely  Useful 

18.8 

25.1 

93.8 

3 

Useful 

47.9 

73.0 

75.0 

4 

Of  Little  use 

24.3 

97.3 

27.1 

5 

Useless 

2.8 

100.1 

2.8 

Since  the  median  category  is  3,  the  total  percent  of  responses  in  cate¬ 
gory  3  is  47.9  percent,  the  total  percent  of  responses  in  categories  1  and 
2  combined  is  25.1  percent,  and  the  total  percent  of  responses  in  cate¬ 
gories  4  and  5  is  27.1  percent,  this  distribution  satisfies  criteria 
number  4.  The  recommendation  is,  teach  this  subject  as  an  elective. 

The  distribution  of  responses  for  the  educational  method  for  this 
recommendation  is: 


EDUCATIONAL 

METHOD 


PERCENT  OF  RESPONDENTS 
ELECTIVE 


OJT 

PCE 

UG 

GRAD 


7.6 

26.7 

50.5 

34.3 


An  undergraduate  education  is  identified  as  the  most  preferred  method  for 
acquiring  competence  in  this  subject.  The  criteria  also  identify  a  grad¬ 
uate  education  as  an  appropriate  method. 

(9)  Research  and  Development  Management.  The  distribution  of  respon¬ 
ses  to  the  usefulness  rating  is: 
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Since  Che  median  category  is  2,  and  Che  total  percent  of  responses  in 
categories  4  and  5  combined  is  4.8  percent,  this  distribution  satisfie; 
criteria  number  1.  Therefore,  the  recommendation  is  that  this  subject  be 
included  in  the  core. 

The  distribution  of  responses  for  the  educational  method  for  this 
recommendation  is: 

EDUCATIONAL  PERCENT  OF  RESPONDENTS 

METHOD  CORE 

OJT  26.5 

PCE  76.9 

UG  10.9 

GRAD  10.9 

PCE  is  identified  as  the  most  preferred  method  for  acquiring  competence 

in  this  subject.  No  other  educational  method  met  the  remaining  criteria. 

After  applying  the  decision  rules  discussed  in  chapter  two,  all  the 
subjects  discussed  in  this  section,  except  one,  were  recommended  to  be 
taught  in  a  cost  analysis  educational  program.  This  alone  indicates  that 
chere  is  concern  about  the  education  of  Air  Force  cost  analysts.  It  is 
quite  evident  that  some  action  is  required  to  improve  the  educational  com¬ 
petence  of  Air  Force  cost  analysts.  The  next  section  evaluates  the 
current  Professional  Continuing  Education  (PCE)  program  offered  to  Air 
Force  cost  and  price  analysts  to  determine  the  usefulness  of  these  courses 
to  current  analysts. 

Rating  of  AFIT  PCE  Courses 

This  section  describes  the  usefulness  ratings  given  to  existing  AFIT 
PCE  courses  related  to  cost  analysis.  A  detailed  description  of  the  cour¬ 
ses  in  this  section  is  provided  in  the  questionnaire  in  Appendix  A.  The 
course  name  and  number  as  it  appears  in  the  questionnaire  is  used  to  iden¬ 
tify  the  eight  courses  in  this  section. 
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(26)  Quantitative  Decisionmaking.  The  distributut ion  of  responses  to 
the  usefulness  rating  is: 


CATEGORY 

RATING 

PERCENT 

RESPONSE 

CUMULATIVE 
1  to  5 

PERCENT  RESPONSE 
5  to  1 

1 

Critical 

8.3 

8.3 

100.2 

2 

Extremely 

Useful 

35.9 

44.2 

91.9 

3 

Useful 

36.6 

80.8 

56.0 

4 

Of  Little 

Use 

16.6 

97.4 

19.4 

5 

Useless 

2.8 

100.2 

2.8 

Since  the  median  category  is  3,  the  total  percent  of  responses  in  cate¬ 
gories  1  and  2  combined  is  44.2  percent,  and  the  total  percent  of  respon¬ 
ses  in  categories  1,  2  and  3  combined  is  80.8  percent,  this  distribution 
satisfies  criteria  number  2.  Therefore,  the  recommendation  is  that  this 
subject  be  included  in  the  core. 

The  distribution  of  responses  for  the  educational  method  for  this 
recommendation  is: 

EDUCATIONAL  PERCENT  OF  RESPONDENTS 


METHOD 

CORE 

OJT 

8.3 

PCE 

28.3 

UG 

46.2 

GRAD 

39.3 

An  undergraduate  education  is  identified  as  the  most  preferred  method  for 


acquir ing 

comptence 

in  this 

subject.  The 

criteria  also 

identify  a  grad- 

uate  education  a3  an 

appropriate  method. 

(27) 

Seminar  in 

Cos t  Analysis.  The 

distribution  of 

responses  to  the 

usefulness 

rating  is 

PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RAT ING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

43.5 

43.5 

100.0 

2 

Extremely 

Useful 

40.1 

83.6 

56.5 

3 

Usefu  1 

11.6 

95.2 

16.4 

4 

Of  Little 

Use 

4.1 

99.3 

4.8 

5 

Useless 

0.  7 

100.0 

0.7 
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EDUCATIONAL 

METHOD 


PERCENT  OF  RESPONDENTS 
CORE 


OJT 

27.2 

PCE 

38.1 

UG 

52.4 

GRAD 

10.9 

An  undergraduate  education  is  identified  as  the  most  preferred  method  for 
acquiring  competence  in  this  subject.  The  criteria  also  identify  PCE  as 
an  appropriate  method. 

(25)  Research  Methods.  The  distribution  of  responses  to  the  use¬ 
fulness  rating  is: 

PERCENT  CUMULATIVE  PERCENT  RESPONSE 


CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

24.0 

24.0 

100.1 

2 

Extremely 

Useful 

45.2 

69.2 

76.1 

3 

Useful 

24.0 

93.2 

30.9 

4 

Of  Little 

Use 

6.2 

99.4 

6.9 

5 

Useless 

0.7 

100.1 

0.7 

Since  the 

median  category  is 

2,  and  the 

total  percent  of 

responses 

categories  4  and  5  combined  is  6.9  percent,  this  distribution  satisfies 
criteria  number  1.  Therefore,  the  recommendation  is  that  this  subject  be 
included  in  the  core. 

The  distribution  of  responses  for  the  educational  method  fc r  this 
recommendation  is: 


EDUCATIONAL 

METHOD 


PERCENT  OF  RESPONDENTS 
CORE 


OJT 

PCE 

UG 

GRAD 


22.6 

42.5 

43.8 

19.2 


The  mode  identified  an  undergraduate  education  as  the  preferred  method  for 
acquiring  competence  in  this  subject.  The  criteria  also  identify  PCE  as 
an  appropriate  method. 
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Since  the  median  category  is  1,  and  the  total  percent  of  responses  in 
categories  4  and  5  combined  is  2.8  percent,  this  distribution  satisfies 
criteria  number  1.  Therefore,  the  recommendation  is  that  this  subject  be 
included  in  the  core. 

The  distribution  of  responses  for  the  educational  method  for  this 
recommendation  is: 


EDUCATIONAL 

METHOD 


PERCENT  OF  RESPONDENTS 
CORE 


OJT 

PCE 

UG 

GRAD 


17.1 

56.2 
37.0 
26.0 


PCE  is  identified  as  the  most  preferred  method  for  acquiring  competence 
in  this  subject.  The  criteria  also  identify  an  undergraduate  education  as 
an  appropriate  method. 


(24)  Technical  Communications.  The  distribution  of  responses  to  the 
usefulness  rating  is: 

PERCENT  CUMULATIVE  PERCENT  RESPONSE 


CATEGORY 

RAT ING 

RESPONSE 

1  to  5 

5  to  1 

1 

Cr it ical 

34.0 

34.0 

100.0 

2 

Extremely  Useful 

38.8 

72.8 

66.0 

3 

Useful 

21.1 

93.9 

27.2 

4 

Of  Little  Use 

6.1 

100.0 

6.1 

5 

Useless 

0.0 

100.0 

0.0 

Since  the 

median  category  is 

2,  and  the 

total  percent  of 

responses 

categories  4  and  5  combined  is  6.1  percent,  this  distribution  satisfies 
criteria  number  1.  Therefore,  the  recommendation  is  that  this  subject  be 
included  in  the  core. 

The  distribution  of  responses  for  the  educational  method  for  this 
recommendation  is: 
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An  undergraduate  education  is  identified  as  the  most  preferred  method  for 
acquiring  competence  in  this  subject.  The  criteria  also  identify  PCE  as 


an  appropriate  method. 


(22) 

Regress  ion 

II.  The 

:  distribution 

of  responses 

to  the  usefulness 

rating  is 

PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Cr it ical 

29.9 

29.9 

100.0 

2 

Extremely 

Useful 

35.4 

65.3 

70.1 

3 

Useful 

25.0 

90.3 

34.7 

4 

Of  Little 

Use 

9.7 

100.0 

9.7 

5 

Useless 

0.0 

100.0 

0.0 

Since  the  median  category  is  2,  and  the  total  percent  of  responses  in 
categories  4  and  5  combined  is  9.7  percent,  this  distribution  satisfies 
criteria  number  1.  Therefore  the  recommendation  is  that  this  subject  be 
included  in  the  core. 

The  distribution  of  responses  for  the  educational  method  is: 

EDUCATIONAL  PERCENT  OF  RESPONDENTS 


METHOD 

CORE 

OJT 

6.9 

PCE 

43.8 

UG 

42.4 

GRAD 

34.0 

PCE  i3  identified  as  the  most  preferred  method  for  acquiring  competence 
in  this  subject.  However,  both  undergraduate  and  graduate  education  are 
also  identified  by  the  criteria  as  appropriate  methods. 


(23)  Quantitative  Analysis.  The  distribution  of  responses  to  the 


use  fu lness 

rating  is: 

PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

57.5 

57.5 

100.0 

2 

Extremely  Useful 

27.4 

84.9 

42.5 

3 

Useful 

12.3 

97.2 

15.1 

4 

Of  Little  Use 

2.1 

99.3 

2.8 

5 

Useless 

0.7 

100.0 

0.7 
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categories  4  and  5  combined  is  7.6  percent,  this  distribution  satisfies 
criteria  number  1.  Therefore,  the  recommendation  is  that  this  subject  be 
included  in  the  core. 


The  distribution  of  responses  for  the  educational  method  is: 

EDUCATIONAL  PERCENT  OF  RESPONDENTS 

METHOD  CORE 


OJT 


6.2 


PCE 

UG 

GRAD 


32.2 

57.5 

28.8 


An  undergraduate  education  is  identified  as  the  most  preferred  method  for 


acquiring 

competence  in  this 

subject.  The  criteria  also 

identify  PCE  as 

an  appropriate  method. 

(21) 

Regression  I.  The 

distribution  of 

response  to 

the  usefulness 

rating  is: 

PERCENT 

CUMULATIVE  PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

41.4 

4.4 

100.0 

2 

Extremely  Useful 

36.6 

78.0 

58.6 

3 

Useful 

17.9 

95.9 

22.0 

4 

Of  Little  Use 

3.4 

99.3 

4.1 

5 

Useless 

0.7 

100.0 

0.7 

Since  the 

median  category  is 

2,  and  the  total 

percent  of 

responses  in  cat 

egories  4 

and  5  combined  is  > 

4.1  percent,  this 

distribution  satisfies  cri- 

teria  number  1.  Therefore,  the  recommendation  is  that  this  subject  be 
included  in  the  core. 

The  distribution  of  responses  for  the  educational  methods  for  this 
recommendation  is: 


EDUCATIONAL 

PERCENT  OF  RESPONDENTS 

METHOD 

CORE 

OJT 

7.6 

PCE 

44.8 

UG 

55.9 

GRAD 

22.8 
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(19)  Introductory  Statistics.  The  distribution  of  responses  to  the 
usefulness  rating  is: 


CATEGORY 

RATING 

PERCENT 

RESPONSE 

CUMULATIVE 

1  to  5 

PERCENT  RESPONSE 
5  to  1 

1 

Critical 

37.0 

37.0 

100.0 

2 

Extremely  Useful 

35.6 

72.6 

63.0 

3 

Useful 

21.2 

93.8 

27.4 

4 

Of  Little  Use 

5.5 

99.3 

6.2 

5 

Useless 

0.7 

100.0 

0.7 

Since  the  median  category  is  2,  and  the  total  percent  of  responses  in 


categories  4  and  5  combined  is  6.2  percent,  this  distribution  satisfies 
criteria  number  1.  Therefore,  the  recommendation  is  that  this  subject  be 
included  in  the  core. 

The  distribution  of  responses  for  the  educational  method  for  this 
recommendation  is: 


EDUCATIONAL 

METHOD 


PERCENT  OF  RESPONDENTS 
CORE 


OJT 

PCE 

UG 

GRAD 


7.5 

20.5 

84.9 

10.3 


An  undergraduate  education  is  indentified  as  the  most  preferred  method  for 
acquiring  competence  in  this  subject.  No  other  educational  method  met 


remaining  criteria. 

(20)  Managerial  Statistics.  The  distribution  of  responses  to  the 
usefulness  rating  is: 

PERCENT  CUMULATIVE  PERCENT  RESPONSE 


CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

32.2 

32.2 

100.1 

2 

Extreme ly 

Useful 

41.8 

74.0 

67.9 

3 

Useful 

18.5 

92.5 

26.1 

4 

Of  Little 

Use 

6.2 

98.7 

7.6 

5 

Useless 

1.4 

100.1 

1.4 

S  ince  the 

median  category  is 

2,  and  the  total 

percent  of 

responses 
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Since  the  median  category  is  3,  the  total  percent  of  responses  in  cate¬ 
gories  1  and  2  combined  is  39.6  percent,  and  the  total  percent  of  respon¬ 
ses  in  categories  1,  2  and  3  combined  is  82.7  percent,  this  distribution 
satisfies  criteria  number  2.  Therefore,  the  recommendation  is  that  this 
subject  be  included  in  the  core. 

The  distribution  of  responses  for  the  educational  method  for  this 
recommendation  is: 


EDUCATIONAL 

METHOD 


PERCENT  OF  RESPONDENTS 
CORE 


OJT 

PCE 

UG 

GRAD 


23.6 

41.0 

56.3 

5.6 


An  undergraduate  education  is  identified  as  the  most  preferred  method  for 
acquiring  competence  in  this  subject.  The  criteria  also  identify  PCE  as 
an  appropriate  method. 

(18)  COBOL  Programming.  The  distribution  of  responses  to  the  use¬ 
fulness  rating  is: 


PERCENT  CUMULATIVE  PERCENT  RESPONSE 


CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

0.0 

0.0 

100.0 

2 

Extremely  Useful 

11. 1 

11.1 

100.0 

3 

Useful 

38.5 

49.6 

88.9 

4 

Of  Little  Use 

43.0 

92.6 

50.4 

5 

Useless 

7.4 

100.0 

7.4 

Since  the 

median  category  is 

4,  and  the 

total  percent 

of  responses 

categories  1  and  2  combined  is  11.1  percent,  this  distribution  satisfies 
criteria  number  5.  Therefore,  the  recommendation  is  that  this  subject  be 
excluded. 


The  distribution  of  responses  for  the  educational  method  for  this 
recommendation  are  not  considered  because  this  subject  should  not  be 
included  in  a  cost  analysis  education  program. 
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(16)  FORTRAN  Programming.  The  distribution  of  responses  to  the  use¬ 
fulness  rating  is: 


CATEGORY 

RATING 

PERCENT 

RESPONSE 

CUMULATIVE 

1  to  5 

PERCENT  RESPONSE 
5  to  1 

1 

Crit ical 

3.5 

3.5 

100.0 

2 

Extremely  Useful 

14.1 

17.6 

96.5 

3 

Useful 

48.6 

66.2 

82.4 

4 

Of  Little  Use 

28.9 

95.1 

33.8 

5 

Useless 

4.9 

100.0 

4.9 

Since  the  median  category  is  3,  and  the  total  percent  of  responses  in  cat¬ 
egories  4  and  5  combined  is  33.8  percent  combined,  this  distribution 
satisfies  criteria  number  3.  Therefore,  the  recommendation  is  that  this 
subject  be  taught  as  an  elective. 

The  distribution  of  responses  for  the  educational  method  for  this 
recommendation  is: 


EDUCATIONAL 

METHOD 


PERCENT  OF  RESPONDENTS 
ELECTIVE 


OJT 

PCE 

UG 

GRAD 


16.0 

41.5 

60.6 
5.3 


An  undergraduate  education  is  identified  as  the  most  preferred  method  for 
acquiring  competence  in  this  subject.  The  criteria  also  identify  PCE  as 
an  appropriate  method. 

(17)  Basic  Programming.  The  distribution  of  responses  to  the  use¬ 
fulness  rating  is: 

PERCENT  CUMULATIVE  PERCENT  RESPONSE 


CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

6.3 

6.3 

100.1 

2 

Extremely  Useful 

33.3 

39.6 

93.8 

3 

Useful 

43.1 

82.7 

60.5 

4 

Of  Little  Use 

15.3 

98.0 

17.4 

5 

Useless 

2.1 

100.0 

2.1 
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EDUCATIONAL 

METHOD 


PERCENT  OF  RESPONDENTS 
ELECTIVE 


OJT 

2.1 

PCE 

12.8 

UG 

87.2 

GRAD 

13.8 

An  undergraduate  education  is  identified  as  the  most  preferred  method  for 
acquiring  competence  in  this  subject.  No  other  educational  method  met  the 
remaining  criteria. 

(15)  Linear  Mathematical  Models.  The  distribution  of  responses  to 


the  usefulness  rating  is: 


CATEGORY 

RATING 

PERCENT 

RESPONSE 

CUMULATIVE 
1  to  5 

PERCENT  RESPONSE 
5  to  1 

1 

Critical 

13.0 

13.0 

100.0 

2 

Extremely  Useful 

34.9 

47.9 

87.0 

3 

Useful 

30.8 

78.7 

52.1 

4 

Of  Little  Use 

19.2 

97.9 

21.3 

5 

Useless 

2.1 

100.0 

2.1 

Since  the  median  category  is  3,  the  total  percent  of  responses  to  cate¬ 
gories  1  and  2  combined  is  47.9  percent,  and  the  total  percent  of  respon¬ 
ses  to  categories  1,  2  and  3  combined  is  78.7  percent,  this  distribution 


satisfies  criteria  number  2.  Therefore,  the  recommendation  is  that  this 
subject  be  included  in  the  core. 

The  distribution  of  responses  for  the  educational  method  for  this 
recommendation  is: 


EDUCATIONAL 

METHOD 


PERCENT  OF  RESPONDENTS 
CORE 


OJT 

PCE 

UG 

GRAD 


11.0 

28.1 

61.0 

24.0 


An  undergraduate  education  is  identified  as  the  most  preferred  method  for 
acquiring  competence  in  this  subject.  No  other  educational  method  met  the 
remaining  criteria. 
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Since  Che  median  category  is  3,  Che  total  percent  of  responses  m  catego¬ 
ries  1  and  2  is  37.6  percent  and  the  total  percent  of  responses  in  catego¬ 
ries  1,  2  and  3  combined  is  74.6  percent,  this  distribution  of  responses 
satisfies  criteria  number  2.  Therefore,  the  recommendation  is  that  this 
subject  be  included  in  the  core. 

The  distribution  of  responses  for  the  educational  method  for  this 
recommendation  is: 


PERCENT  OF  RESPONDENTS 
CORE 


EDUCATIONAL 

METHOD 

OJT 

PCE 

UG 

GRAD 


An  undergraduate  education  is  identified  as  the  most  preferred  method  for 
acquiring  competence  in  this  subject.  No  other  educational  method  met  the 
remaining  criteria. 

(14)  Calculus .  The  distribution  of  responses  to  the  usefulness 
rating  is: 


PERCENT 


CUMULATIVE  PERCENT  RESPONSE 


SR 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

' 

1 

Critical 

5.6 

5.6 

100.1 

2 

Extremely  Useful 

21.0 

26.6 

94.5 

3 

Useful 

39.2 

65.8 

73.5 

4 

Of  Little  Use 

29.4 

95.2 

34.3 

r 

5 

Useless 

4.9 

100.1 

4.9 

Since  the 

median  category  is 

3, 

and  the  total 

percent  of  responses  in 

i 

i 

egories  4 

and  5  combined  is 

34. 

3  percent,  this 

distribution 

satisfies 

i  i 

f 

teria  number  3.  Therefore, 

the 

recommendat ion 

is  that  this 

subject  be 

taught  as  an  elective. 

The  distribution  of  responses  for  the  educational  method  for  this 
recommendation  is: 
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Both  PCE  and  an  undergraduate  education  were  recommended  as  appropriate 
methods  for  acquiring  competence  in  this  subject  whether  it  was  included 
in  the  core  or  taught  as  an  elective.  However,  the  recommendations  in 
each  case  were  based  on  different  criteria. 

(12)  Defense  Production  Management.  The  distribution  of  responses  to 
the  usefulness  rating  is: 

PERCENT  CUMULATIVE  PERCENT  RESPONSE 


CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

13.8 

13.8 

100.1 

2 

Extremely  Useful 

33.1 

46.9 

86.3 

3 

Useful 

35.9 

82.8 

53.2 

4 

Of  Little  Use 

15.2 

98.0 

17.3 

5 

Useless 

2.1 

100.1 

2.1 

Since  the  median  category  is  3,  and  the  total  percent  of  responses  in  cat¬ 
egories  1,  2  and  3  combined  is  82.8  percent,  this  distribution  satisfies 
criteria  number  2.  The  recommendation  is  that  this  subject  be  in  the  core. 
The  distribution  of  responses  for  the  educational  method  is: 

EDUCATIONAL  PERCENT  OF  RESPONDENTS 


METHOD 

CORE 

OJT 

35.9 

PCE 

66.2 

UG 

6.9 

GRAD 

4.8 

PCE  is  identified  as  the  most  preferred  method  for  acquiring  competence 
in  this  subject.  The  criteria  also  identify  OJT  as  an  appropriate  method. 

(13)  Matricies  and  Linear  Algebra.  The  distribution  of  responses  to 
the  usefulness  rating  is: 


PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

12.3 

12.3 

100.0 

2 

Extremely  Useful 

25.3 

37.6 

87.7 

3 

Useful 

37.0 

74.6 

62.4 

4 

Of  Little  Use 

23.3 

97.9 

25.4 

5 

Useless 

2.1 

100.0 

2.1 
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The  distribution  of  responses  for  the  educational  method  for  this 


recommendation  is: 


EDUCATIONAL 

METHOD 


PERCENT  OF  RESPONDENTS 
CORE 


OJT 

PCE 

UG 

GRAD 


33.1 

41.2 
31.1 
20.9 


PCE  is  identified  as  the  most  preferred  method  for  acquiring  competence 


in  this  subject.  The  criteria  also  identify  OJT  and  an  undergraduate  edu 


cation  as  preferred  methods. 

(11)  Production  Management.  The 
usefulness  rating  is: 


PERCENT 


CATEGORY 

RATING 

RESPONSE 

1 

Critical 

6.9 

2 

Extremely  Useful 

22.8 

3 

Us  e  fu  1 

50.3 

4 

Of  Little  use 

17.2 

5 

Useless 

2.8 

distribution  of  responses  to  the 


CUMULATIVE  PERCENT  RESPONSE 
1  to  5  5  to  1 


6.9 

100.0 

29.7 

93.1 

80.0 

70.3 

97.2 

20.0 

100.0 

2.8 

Since  the  total  number  of  responses  in  category  3  is  50.3  percent,  the 
total  percent  of  responses  in  categories  1  and  2  combined  is  29.7  percent 
and  the  total  percent  of  responses  in  categories  4  and  5  is  20.0  percent, 
this  distribution  satisfies  criteria  number  7.  Therefore,  the  primary 
recommendation  is  that  this  subject  be  included  in  the  core,  and  the 
secondary  recommendation  is  that  it  be  taught  as  an  elective. 

The  distribution  of  responses  for  the  educational  method  for  each  of 
the  two  recommendations  is: 


EDUCATIONAL 

METHOD 


PERCENT  OF  RESPONDENTS 
CORE  ELECTIVE 


OJT 

25.5 

23.3 

PCE 

33.8 

37.1 

UG 

35.9 

36.2 

GRAD 

22.8 

25.0 
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PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

15.6 

15.6 

100.1 

2 

Extremely  Useful 

21.8 

37.4 

84.5 

3 

Useful 

46.3 

83.7 

62.7 

4 

Of  Little  use 

15.0 

98.7 

16.4 

5 

Useless 

1.4 

100.1 

1.4 

Since  the  median  category  is  3,  the  total  percent  of  responses  in  cate¬ 
gories  1  and  2  combined  is  37.4  percent,  and  the  total  percent  of  respon¬ 
ses  in  categories  1,  2  and  3  combined  is  83.7,  this  distribution  satisfies 
criteria  number  2.  Therefore,  the  recommendation  is  that  this  subject  be 
included  in  the  core. 

The  distribution  of  responses  for  the  educational  method  is: 


EDUCATIONAL 

METHOD 


PERCENT  OF  RESPONDENTS 
CORE 


OJT 

PCE 

UG 

GRAD 


35.4 

63.3 

2.7 

10.9 


PCE  is  identified  as  the  most  preferred  method  for  acquiring  competence 


in  this  subject.  The  criteria  also  identify  OJT  as  an  appropriate  method. 


(10)  Management  Information  Systems.  The  distribution  of  responses 


to  the  usefulness  rating  is: 


PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

14.9 

14.9 

100.1 

2 

Extremely  Useful 

39.2 

54.1 

85.2 

3 

Useful 

33.1 

87.2 

46.0 

4 

Of  Little  Use 

11.5 

98.7 

12.9 

5 

Useless 

1.4 

100.1 

1.4 

Since  the  median  category  is  2,  and  the  total  percent  of  responses  in 
categories  4  and  5  is  12.9  percent,  this  distribution  satisfies  criteria 
number  1.  Therefore,  the  recommendation  is  that  this  subject  be  included 
in  the  core. 
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For  every  course  evaluated,  the  distribution  of  responses  has  a  median 
category  of  2,  and  the  total  percent  of  responses  in  categories  4  and  5 
combined  is  less  than  20  percent,  vhich  satisfies  criteria  number  1. 


Therefore,  it  is  recommended  that  every  course  be  included  in  the  core. 

The  distribution  of  responses  to  the  usefulness  rating  for  each  course  i: 

(28)  QMT  1"0  Principles  of  Contract  Pricing. 

PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

21.2 

21.2 

99.9 

2 

Extremely  Useful 

43.8 

65.0 

78.7 

3 

Useful 

28.1 

93.1 

34.9 

4 

Of  Little  Use 

6.8 

99.9 

6.8 

5 

Useless 

0.0 

99.9 

0.0 

(29) 

QMT  345  Introductory  Quantitative 

Analysis. 

PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

'•'.7 

39.7 

100.0 

2 

Extremely  Useful 

42.5 

82.2 

60.3 

3 

Useful 

15.1 

97.3 

17.8 

4 

Of  Little  Use 

2.7 

100.0 

2.7 

5 

Useless 

0.0 

100.0 

0.0 

(30) 

QMT  550  Advanced  Quantitative  Methods. 

PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

27.6 

27.6 

100.0 

2 

Extremely  Useful 

39.3 

66.9 

72.4 

3 

Useful 

24.1 

91.0 

33.1 

4 

Of  Little  Use 

9.0 

100.0 

9.0 

5 

Useless 

0.0 

100.0 

0.0 

(31) 

QMT  551  Advanced  Cost  and  Economic 

Analys is . 

PERCENT 

CUMULAT IVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Cr it ical 

23.2 

23.2 

99- 9 

2 

Extremely  Useful 

42.3 

65.5 

76.7 

3 

Useful 

24.6 

90.1 

34.4 

4 

Of  Little  Use 

7.0 

97.1 

9.8 

5 

Useless 

2.8 

99.9 

2.8 
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(32)  QMT  180  Learning  Curve  Analysis. 


PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

39.6 

39.6 

100.1 

2 

Extremely  Useful 

30.6 

70.2 

60.5 

3 

Useful 

22.9 

93.1 

29.9 

4 

Of  Little  Use 

4.9 

98.0 

7.0 

5 

Use  less 

2.1 

100.1 

2.1 

(33) 

QMT  353  Introduction  to  Life 

Cycle 

Cost  Management. 

PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

19.0 

19.0 

100.0 

2 

Extremely  Useful 

45.8 

64.8 

81.0 

3 

Useful 

31.0 

95.8 

35.2 

4 

Of  Little  Use 

3.5 

99.3 

4.2 

5 

Useless 

0.7 

100.0 

0.7 

(34) 

QMT  540  Advanced  ] 

Pricing  Methods. 

PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Cr it ical 

14.2 

14.2 

100.0 

2 

Extremely  Useful 

36.2 

50.4 

85.8 

3 

Useful 

39.0 

89.4 

49.6 

4 

Of  Little  Use 

9.9 

99.3 

10.6 

5 

Useless 

0.7 

100.0 

0.7 

(35) 

SYS  362  Cost  Schedule  Control 

System  Criteria. 

PERCENT 

CUMULATIVE 

PERCENT  RESPONSE 

CATEGORY 

RATING 

RESPONSE 

1  to  5 

5  to  1 

1 

Critical 

22.7 

22.7 

100.0 

2 

Extremely  Useful 

56.7 

77.3 

3 

Useful 

36.2 

92.9 

43.3 

4 

Of  Little  Use 

7.1 

100.0 

7.1 

5 

Useless 

0.0 

100.0 

/.  n 


[iWil 


Written  Comments 


There  were  a  number  of  written  comments  received  in  answer  to  the 
open-ended  question  at  the  end  of  the  questionnaire.  This  question  asked 
the  respondent  to  briefly  describe  how  they  thought  the  educational  com¬ 
petence  of  all  Air  Force  cost  analysts  could  be  improved.  The  comments 
centered  around  five  general  areas  of  concern,  each  of  which  will  be  indi¬ 
vidually  analyzed. 

The  qualifications,  or  lack  thereof,  of  people  entering  the  field  was 
a  major  concern  of  many  of  the  respondents.  Three  deficiences  were  iden¬ 
tified  which,  if  corrected,  would  greatly  enhance  the  proficiency  of  the 
work  force.  First,  an  undergraduate  degree  in  a  quantitative  discipline 
should  be  a  prerequisite  for  entering  the  cost  analysis  career  field.  Not 
only  would  the  curriculum  include  many  of  the  techniques  required  of  a  cost 
analyst,  but  possession  of  the  degree  also  indicates  that  the  individual 
has  the  capacity  to  learn  new  techniques  as  required.  Second,  every  ana¬ 
lyst  should  have  a  strong  math  background.  Math  represents  both  the 
language  and  the  foundation  of  quantitative  analysis.  Without  a  math 
background,  an  analyst's  ability  to  expand  his  knowledge  is  severely 
limited.  Finally,  an  analyst  needs  to  know  something  about  the  technical 
and  engineering  aspects  of  the  systems  he  is  estimating.  This  knowledge 
provides  him  with  a  viable  tool  for  understanding  the  complex  relationship 
between  technical  specifications  and  resource  requirements.  This  qualifi¬ 
cation  also  requires  that  the  analyst  specialize  in  estimating  certain 
categories  of  systems. 

OJT  provides  the  opportunity  to  apply  newly  learned  techniques  to  real 
world  problems  and  acquire  the  experience  which  separates  the  apprentice 
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from  the  senior  analyst.  Many  of  the  respondents  felt  that  OJT  was  absolu¬ 
tely  necessary  for  an  analyst  to  become  fully  qualified.  However,  the 
haphazard  approach  to  OJT  was  highly  criticized.  Again,  three  problems 
were  identified  which  resulted  in  poor  OJT  programs.  First,  many  respon¬ 
dents  complained  of  a  lack  of  formal  OJT  programs.  Where  the  programs 
existed,  many  lacked  a  comprehensive  plan  which  identified  specific  educa¬ 
tional  goals.  Consequently,  the  lack  of  a  program  or  a  specific  plan 
resulted  in  many  deficiencies  among  Air  Force  cost  analysts.  Second,  edu¬ 
cational  goals  were  seldom  accomplished  in  a  systematic  manner  over  a 
reasonable  period  of  time  because  of  conflicting  job  requirements.  This 
failure  is  frequently  masked  by  the  absence  of  a  formal  OJT  program. 

Instead  of  assigning  work  to  create  opportunities  for  accomplishing  educa¬ 
tional  goals,  immediate  job  requirements  determined  work  assignments 
without  regard  for  their  value  to  the  OJT  program.  Finally,  a  good  OJT 
program  requires  that  a  qualified  individual  spend  time  working  with  the 
trainee  in  an  instructor  role.  Frequently,  the  workload  of  a  qualified 
analyst  leaves  him  little  time  to  spend  instructing  a  trainee.  Also,  many 
qualified  analysts  lack  the  skills  necessary  to  effectively  communicate  in 
their  role  as  an  instructor.  Consequently,  the  trainee  is  frequently  given 
the  responsibility  to  train  himself. 

Respondents  also  made  several  observations  concerning  the  PCE  program. 

A  general  category  of  complaints  centered  around  the  availability  of  course 
offerings.  The  lack  of  ability  to  send  the  right  person  to  the  right 
course  at  the  right  time  was  a  major  concern.  This  problem  addresses  the 
issue  of  frequency  of  course  offerings,  the  schedule  of  offerings,  and  the 
quota  system  for  allocating  student  slots.  This  problem,  while  important, 
goes  beyond  the  scope  of  this  research.  Many  respondents  also  expressed 


Che  need  for  recurring  continuing  educacion  to  gain  periodic  exposure  to 
new  developments  and  techniques  and  to  share  experiences  with  other  senior 
analysts.  Annual  attendance  was  recommended  by  most  respondents. 

Many  of  the  respondents  seem  to  believe  that  PCE  courses  should  address 
only  chose  skills  or  knowledge  specifically  required  to  perform  the  job  to 
which  the  analyst  is  assigned.  This  belief  was  reflected  in  the  recommen¬ 
dations  that  courses  should  be  structured  around  specific  job  requirements, 
that  separate  courses  should  be  developed  to  present  the  operating  command 
versus  systems  command  perspective,  and  that  analysts  should  be  taught  only 
the  basic  techniques  early  in  their  career  while  the  more  advanced  tech¬ 
niques  should  be  taught  as  needed.  There  was  also  an  expressed  concern 
that  the  courses  should  concentrate  on  "real  world"  applications,  with  an 
emphasis  on  case  studies,  as  opposed  to  "theory". 

A  key  issue  which  dismayed  a  number  of  respondents  was  the  professiona¬ 
lism  of  Air  Force  cost  analysts.  They  identified  having  a  broad  perspec¬ 
tive  as  one  of  the  key  characteristics  of  a  profess ional ,  and  seemed 
convinced  that  a  narrow  job  focus  limited  the  ability  of  an  analyst  to  per¬ 
form  complex  estimating  tasks.  Consequently,  they  recommended  free 
exchange  of  ideas  and  periodic  job  rotation  as  means  for  enhancing  the  pro¬ 
fessional  development  of  the  analyst.  Another  concern  has  to  do  with  the 
lack  of  mandatory  qualifications  which  must  be  met  before  an  individual  is 
recognized  as  a  fully  qualified  cost  analyst.  The  only  prerequisite  for  a 
military  cost  analyst  is  that  he  spend  18  months  on  the  job.  The  lack  of  a 
cost  analysis  civilian  career  field  means  that  there  are  no  established 
qualifications  for  identifying  qualified  civilian  cost  analysts. 

Respondents  suggested  that  mandatory  PCE  courses  and/or  professional  cer¬ 
tification  should  be  required  before  a  cost  analyst  is  recognized  as  fully 
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qualified.  A  final  concern  is  that  there  is  an  inadequate  career 
progression.  For  the  military,  the  Lack  of  a  reasonable  number  of  high 
grade  positions  forces  the  analyst  to  leave  the  cost  analysis  career  field 
upon  being  promoted.  For  the  civilian,  career  progression  is  not  as  signi¬ 
ficant  a  problem.  However,  the  establishment  of  a  distinct  civilian  cost 
analysis  career  field  was  still  considered  desirable  in  order  to  enhance 
the  professional  status  of  the  analysts. 

Throughout  all  of  the  comments,  one  idea  consistently  emerged  as  a 
central  theme.  None  of  the  problems  identified  in  this  research  are  going 
to  be  solved  without  a  direct  and  concerted  effort  on  the  part  of  manage¬ 
ment.  This  means  that  management  must  recognize  the  importance  of  deve¬ 
loping  a  career  cost  analyst  to  his  fullest  potential  and  be  prepared  to 
make  the  decisions  necessary  to  achieve  that  goal.  The  long  term  benefits 
to  the  Air  Force  exceed  the  short  term  costs  incurred  by  the  supervisor  for 
participating  in  this  effort. 
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IV.  Cone lus ion 3 /Recommendat ions 


This  research  had  the  objective  of  identifying  the  educational 
requirements  that  experienced  Air  Force  cost  analysts  thought  were  most 
useful  for  adequate  job  performance.  An  objective  evaluation  of  the  mail 
survey  utilizing  the  decision  rules  in  chapter  two  resulted  in  the 
following  Air  Force  cost  analyst  educational  requirements  list,  presented 
by  each  educational  method.  In  the  lists  that  follow  the  number  contained 
in  the  parentheses  signifies  the  order  in  which  the  subject  appeared  in 
the  quesionnaire.  The  "*"  identifies  those  subjects  recommended  to  be 
taught  by  more  than  one  method. 

Those  subjects  identified  to  be  taught  by  OJT  are  listed  below: 

Core: 

1.  Federal  Financial  Management  (6)* 

2.  Research  and  Development  Management  (9)* 

3.  Management  Information  Systems  (10)* 

4.  Defense  Production  Management  (12)* 

Those  subjects  identified  to  be  taught  by  PCE  are  listed  below: 

Core : 

1.  Technological  Forecasting  (5) 

2.  Federal  Financial  Management  (6)* 

3.  Research  and  Development  Management  (9)* 

4.  Management  Information  Systems  (10)* 

3.  Defense  Production  Managment  (12)* 

6.  Basic  Programming  (17)* 

7.  Managerial  Statistics  (20)* 

8.  Regression  I  (21)* 

9.  Regression  II  (22)* 

10.  Quantitative  Analysis  (23)* 

11.  Technical  Communications  (24)* 

12.  Research  Methods  (25)* 

13.  Seminar  in  Cost  Analysis  (27) 

Core/elective: 

14.  Production  Management  (11)* 
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Elective: 


15.  FORTRAN  Programming  (16)* 

Those  subjects  identified  to  be  taught  as  part  of  an  undergraduate 
degree  program  are  listed  below: 

Core: 

1.  Financial  and  Cost  Accounting  (1) 

2.  Managerial  Accounting  (2) 

3.  Macroeconomics  (4) 

4.  Management  Information  Systems  (10)* 

5.  Matrices  and  Linear  Algebra  (13) 

6.  Linear  Mathematical  Models  (15) 

7.  Basic  Programming  (17)* 

8.  Introductory  Statistics  (19) 

9.  Managerial  Statistics  (20)* 

10.  Regression  I  (21)* 

11.  Regression  II  (22)* 

12.  Quantitative  Analysis  (23)* 

13.  Technical  Communications  (24)* 

14.  Research  Methods  (25)* 

15.  Quantitative  Decisionmaking  (26)* 

Core/elective: 

16.  Microeconomics  (3) 

17.  Production  Management  (11)* 

Elective: 

18.  Managerial  Finance  (7) 

19.  Management  and  Behavior  in  Organizations  (8)* 

^20.  Calculus  (14) 

*21.  FORTRAN  Programming  (17)* 

Those  subjects  identified  to  be  taught  as  part  of  a  graduate  program 
are  listed  below: 

Core : 

1.  Regression  II  (22)* 

2.  Quantitative  Decisionmaking  (26)* 

Elective: 

3.  Management  and  Behavior  in  Organizations  (8)* 

Some  subjects  were  recommended  for  more  than  one  educational  method.  Onl 
one  subject  had  a  median  usefulness  rating  of  one  (Quantitative  Analysis) 


Also,  only  one  subject  was  not  recommended  to  be  taught  (COBOL  Programming). 

AFIT  cost  analysis  related  PCE  courses  were  also  evaluated  for  their 
usefulness  to  experienced  cost  analysts.  The  recommendation  was  that  all 
of  the  existing  PCE  courses  be  included  as  mandatory  requirements  in  the 
cost  analysis  PCE  program.  These  courses  are  listed  below: 

1.  QMT  170  Principles  of  Contract  Pricing  (28) 

2.  QMT  345  Introductory  Quantitative  Analysis  (29) 

3.  QMT  550  Advanced  Quantitative  Methods  (30) 

4.  QMT  551  Advanced  Cost  and  Economic  Analysis  (31) 

5.  QMT  180  Learning  Curve  Analysis  (32) 

6.  QMT  353  Introduction  to  Life  Cycle  Cost  Management  (33) 

7.  QMt  540  Advanced  Pricing  Methods  (34) 

8.  SYS  362  Cost  Schedule  Control  System  Criteria  (35) 

Written  comments  included  in  the  questionnaire  centered  around  the 
issues  of  qualifications  for  cost  analysts,  OJT  programs,  PCE  programs  and 
course  content,  professionalism  of  the  work  force,  and  the  need  for  mana¬ 
gement  attention  to  these  problems. 

The  educational  requirements  listed  above  can  be  used  by  Air  Force 
military  and  civilian  cost  analyst  career  managers  for  evaluating  poten¬ 
tial  analysts,  for  judging  the  promotion  potential  of  current  analysts, 
for  evaluating  current  analysts  for  future  assignments,  and  for  evaluating 
a  current  analyst's  need  or  request  for  higher  education.  The  require¬ 
ments  can  also  be  used  by  the  analysts  themselves  for  guiding  their  educa¬ 
tion  and  professional  development.  In  addition,  those  who  actually  train 
and  supervise  the  analysts  can  use  the  list  as  an  aid  for  instituting  an 
on-the-job  training  (OJT)  program,  tailoring  the  list  to  job  specific 
needs . 

The  list  can  also  be  used  by  AFIT  and  other  institutions  of  higher 
learning  for  evaluating  the  appropriateness  of  existing  cost  analysis  edu¬ 
cational  programs.  The  number  of  courses  suggested  as  PCE  indicates  that 
AFIT  should  consider  expanding  their  existing  PCE  program.  The  data 
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suggest  that  this  is  a  preferred  method  for  improving  current  cost  analy¬ 
sis  education.  Finally  the  list  may  be  used  to  explore  the  need  for  an 
undergraduate  cost  analysis  program.  This  idea  of  an  undergraduate  cost 
analysis  curriculum  may  be  a  worthwhile  pursuit  in  light  of  the  substan¬ 
tial  number  of  subjects  recommended  to  be  taught  at  the  undergraduate 
level.  The  institution  of  an  undergraduate  program  at  the  Air  Force 
Academy  and/or  at  the  Air  Force  Institute  of  Technology  is  a  thought  well 
worth  investigating  since  the  Air  Force  has  more  influence  at  these  insti¬ 
tutions  than  it  would  at  convincing  any  civilian  university  to  carry  such 
a  program. 

Recommendations  for  Further  Research 

The  cost  analysis  field  is  fairly  new,  and  interest  in  the  field  is 
growing  daily.  Further  research  to  either  develop  a  formal  Air  Force 
cost  analyst  OJT  program  or  a  detailed  evaluation  of  the  feasibility  of  an 
undergraduate  cost  analysis  curriculum  would  be  areas  of  study  invaluable 
to  the  field.  Also  an  investigation  into  the  means  used  to  acquire  Air 
Force  cost  analysts  and  the  policies  used  to  evaluate  the  adequacy  of 
those  potential  analysts  may  well  be  worth  exploring.  Finally  a  com¬ 
parison  of  worker  attitudes  for  and  against  a  unique  job  series  for  Air 
Force  civilian  cost  analysts  will  too  be  a  subject  of  considerable  impor¬ 
tance  to  all  Air  Force  cost  analysts.  The  cost  analysis  field  is  wide 
open  for  research,  and  hopefully  this  effort  will  not  be  the  last  to 
examine  the  educational  development  or  for  that  matter,  any  development 
within  the  field. 
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Appendix  A:  Interview  Guide  and  Questionnaire 


Interview  Guide 


NAME: 

DATE : 

ORGAN IZ AT ION / POS IT ION : 

EXPERIENCE  (in  terms  of  years,  positions  held,  formal  education): 

PURPOSE  OF  INTERVIEW:  This  is  a  thesis  research  effort  to  determine  the 

educational  requirements  that  are  necessary  for  an 
Air  Force  cost  analyst  to  perform  his  job  effec¬ 
tively. 

CONSIDERING  THE  FOLLOWING  EDUCATIONAL  CATEGORIES,  WHAT  TOPICS,  CONCEPTS, 

OR  TECHNIQUES  SHOULD  A  COST  ANALYST  KNOW: 

MATH  (i.e.  matrix  algebra,  differential/integral  calculus,  etc.) 

STATISTICS  (i.e.  probability  theory,  interval  estimates,  etc.) 

OPERATIONS  RESEARCH/MANAGEMENT  SCIENCE  (linear  programming,  queuing 
theory,  decision  theory,  etc.) 

ACCOUNTING /FINANCE  (inventory  costing,  depreciation  methods,  congressional 
budget  process,  profitability,  liquidity  etc.) 

ECONOMICS  (microeconomics/macroeconomics:  time  value  of  money,  discount¬ 
ing,  rates  of  return,  etc.) 

COMPUTER  SCIENCE  (basic,  fortran,  cobol,  information  systems,  etc.) 

MANAGEMENT  (management  by  objective,  goal  setting,  various  contemporary 
theories  of  management,  etc.) 

TECHNICAL  COMMUNICATIONS  (writing  skills,  oral  skills,  style  and  format  in 
technical  communications,  etc.) 

ENGINEERING  (evaluation  of  design  drawings,  interpreting  military  specifi¬ 
cations,  etc.) 

VARIOUS  OTHER  TOPICS  OF  USE  TO  AIR  FORCE  COST  ANALYSTS 

UNDERGRADUATE  DEGREE  PREFERENCES  FOR  COST  ANALYSTS 

APPLICATION  OF  AFIT  PCE  COURSES  TO  JOBS  AND  OJT  PLANS 
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DEPARTMENT  OF  THE  AIR  FORCE 
AIR  FORCE  INSTITUTE  OF  TECHNOLOGY  (AU) 
WRIGHT-PATTERSON  AIR  FORCE  BASE.  OH  45433 


1  JUN  1984 


REPLY  TO 

attn  of  LSA  (AFIT/GSM/LSY/84S-24) /Lieutenant  Phillip  Perry /AUTOVON  785-7212 
subject  Air  Force  Cost  Analyst  Educational  Requirements  Survey 


to  Experienced  Air  Forcf  Cost  Analysts 

1.  The  attached  questionnaire  was  prepared  by  a  graduate  student  at  the 
Air  Force  Institute  of  Technology,  Wright-Patterson  AFB,  OH.  The  purpose 
of  the  questionnaire  is  to  acquire  data  concerning  an  experienced  Air 
Force  cost  analyst's  perception  of  his  job  and  the  educational  requirements 
that  he  identifies  as  necessary  to  adequately  perform  that  job. 

2.  You  are  requested  to  provide  an  answer  or  comment  for  each  question. 
Headquarters  USAF  Survey  Control  Number  84-42  has  been  assigned  to 
this  questionnaire.  Your  participation  in  this  research  is  voluntary. 

3.  Your  responses  to  the  questions  will  be  held  confidential.  Please 
remove  this  cover  sheet  before  returning  the  completed  questionnaire. 

Your  cooperation  in  providing  this  data  will  be  appreciated  and  will  be 
very  beneficial  in  examining  the  adequacy  of  Air  Force  cost  analysis 
education  efforts.  Please  return  the  completed  questionnaire  in  the  attached 

receipt. 


2  Atch 

1.  Questionnaire 

2 .  Return  Envelope 


within  one  week  after 


MITH,  Colonel,  USAF 
Systems  and  Logistics 
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AIR  FORCE -A  GREAT  WAY  OF  LIFE 


DEPARTMENT  OF  THE  AIR  FORCE 

HEADQUARTERS  UNITED  STATES  AIR  FORCE 
WASHINGTON.  D  C 


1  JUN  1S34 

subject  Completing  Air  Force  Survey  84-42,  Cost  Analysis  Education  Requirements  in 
the  Ai r  Force 

TO  Experienced  Air  Force  Cost  Analysts 

1.  I  would  like  you  to  complete  the  attached  survey  (84-42)  of  Air  Force 
cost  analysts  to  support  our  research  into  the  education  requirements  for 
cost  analysts.  You  were  selected  to  complete  this  survey  because  your 
records  show  you  have  experience  either  as  a  cost  analyst  or  in  a  closely 
related  field.  Your  personal  experience  and  insight  into  what  we  need  in  the 
future  is  valuable  information.  I  appreciate  your  voluntary  participation  in 
this  research  effort;  your  responses  will  be  held  confidential. 

2.  This  research  is  being  conducted  by  Phil  Perry,  a  graduate  student  in  cost 
analysis  at  the  Air  Force  Institute  of  Technology.  The  purpose  of  the  survey 
is  to  gather  opinions  from  experienced  analysts  like  yourself  as  to  what  is 
needed  to  become  an  effective  cost  analyst.  The  results  of  his  research  will 

be  published  in  September,  1984,  as  part  of  his  thesis  on  education  requirements 
for  Air  Force  cost  analysts. 

3.  This  is  your  opportunity  to  impact  the  future  of  cost  analysis  in  the  Air 
Force.  If  you  chose  to  participate  in  this  research  effort  please  answer  each 
question  thoughtfully,  detach  it  from  this  letter,  and  return  it  in  the  envelope 
provided.  If  you  chose  not  to  answer  the  survey  return  the  survey  package  in 
the  envelope. 

4.  Respond  to  this  survey  quickly  (this  week)  as  it  affects  Lt  Perry's  thesis 
effort.  He  cannot  start  his  analysis  until  he  recieves  your  data  input.  You 
may  address  questions  and  surveys  to  Lt  Phil  Perry  (AV  785-7212)  AFIT/LSY 
(84S-24),  WrlgJxtiPatterson  AFB  OH  45433. 


Oirec&r  a!  Cast  tii 
MaupMSt  Ansirs* 


RC«*L  V  JO 

ATTN  OF  ACM 


2  Atch 

1.  Survey  of  Cost  Analysts 

2.  Return  Envelope 


USAF  SCN  84-42 


PRIVACY  ACT  STATEMENT 


In  accordance  with  paragraph  8,  AFR  12-35,  the  following  information  is 
provided  as  required  by  the  Privacy  Act  of  1974: 

a.  Authority: 

(1)  5  U.S.C.  301,  Departmental  Regulations;  and/or 

(2)  10  U.S.C.  8012,  Secretary  of  the  Air  Force,  Powers,  Duties, 
Delegation  by  Compensation;  and/or 

(3)  DOD  Instruction  1100.13,  17  Apr  68,  Surveys  of  Department  of 
Defense  Personnel;  and/or 

(4)  AFR  30-23,  22  Sep  76,  Air  Force  Personnel  Survey  Program. 

b.  Principal  purposes.  The  survey  is  being  conducted  to  collect 
information  to  be  used  in  research  aimed  at  illuminating  and  providing 
inputs  to  the  solution  of  problems  of  interest  to  the  Air  Force  and/or 
DOD. 


c.  Routine  uses.  The  survey  data  will  be  converted  to  information  for 
use  in  research  of  management  related  problems.  Results  of  the  research, 
based  on  the  date  provided,  will  be  included  in  written  master's  theses 
and  may  also  be  included  in  published  articles,  reports,  or  texts. 
Distribution  of  the  results  of  the  research,  based  on  survey  data,  whether 
in  written  form  or  presented  orally,  will  be  unlimited. 

d.  Participation  in  this  survey  is  entirely  voluntary. 

e.  No  adverse  action  of  any  kind  may  be  taken  against  any  individual 
who  elects  not  to  participate  in  any  or  all  of  this  survey. 


"Basic  courses  are  generally  good  -  if  you  can  get  to  them.  While 
recognize  budget  constraints  are  limited  when  all  too  often  emphasis  is 
put  on  letting  the  new  military  member  go  at  the  expense  of  continuing 
development  of  the  civilian  member  of  the  team.  We've  definitely  been 
made  aware  that  we  are  second  class  citizens  although  we  carry  on  the 
major  portion  nf  the  load.  This  is  one  cost  analyst  that  is  tired  of 
paying  for  my  education.  Course  in  all  the  years  at  government  expense  is 
rediculous.  You  can't  even  get  time  off  for  professional  seminars  unless 
you  take  leave.  Those  I  don't  mind  paying  for  but  why  leave?" 

"Applications  of  a  sound  professional  continuing  education  program  needs 
to  be  enforced." 

We  need  a  cost  estimating  handbook." 

"Establish  a  thorough  interdisciplinary  OJT  program  for  C/SCSC  cost  ana¬ 
lysts,  estimator  cost  analysts,  price  analysts,  &  budget  analysts. 

Knowing  each  other's  discipline  would  increase  cross-feed  of  information 
that  is  valuable  and  relevant  to  all  four  disciplines." 

"The  AF1T  program  should  be  expanded  so  that  those  of  use  who  don't  go  to 
the  graduate  program  can't  take  some  of  the  kinds  of  courses  shown  above 
as  part  of  our  continuous  education  requirements." 

"The  courses  offered  by  AFIT  are  not  as  stringent  as  they  could  be  -  make 
it  tougher  on  the  individuals  taking  the  course.  In  this  manner  you  are 
assured  that  the  courses  are  producing  competent  graduates  not  just  check 
marks  on  a  promotion  list!!" 

"The  most  critical  and  difficult  phase  of  cost  estimating  is  getting  a 
good  and  detailed  understanding  of  what  is  being  costed.  Estimators  need 
a  better  understanding  of  technology,  manufacturing  methods,  and  of  the 
hardware  that  make  up  the  weapon  systems  they  estimate.  A  course  exposing 
the  estimators  to  this  type  of  information  is  critically  needed. 

Estimators  coming  out  of  AFIT  are  long  on  theory  and  very  short  on  prac¬ 
tical  applications.  They  need  a  better  feel  for  the  actual  problems 
encountered  in  doing  an  estimate.  The  cost  analysis  seminar  should  focus 
on  actual  experience  by  having  the  students  cio  estimates  -  not  just  run 
the  numbers  but  do  actual  inteviews  with  engineers  and  program  managers, 
hunt  for  data,  choose  the  estimating  methods,  and  present  the  estimate." 

"In  class  training  is  very  difficult  in  the  cost  analysis  field.  Almost 
any  quantitative  course  can  add  to  the  basic  for  becoming  competent  in  the 
cost  analysis  field,  but  none  can  do  the  job  without  adequate  on-the-job 
training.  Every  task  in  estimating  is  different  and  so  many  variables  are 
involved.  The  need  to  expand  the  work  force  to  enable  the  experienced 
analyst  to  have  enough  time  to  spend  with  new  analysts  on  the  job. 

Without  the  up  front  investment  of  time  new  analyst  will  take  longer  to 
become  productive." 

"The  mathematics,  basic  sciences  and  engineering  courses  are  most  effec¬ 
tive  from  the  "university"  environment.  Efforts  by  AFIT  to  make 
mathemat ic  ian/3  tat  is t ic ians/engineers  out  of  people  in  a  few  weeks  does 
not  work.  Cost  analysis  is  still  best  learned  through  OJT  under  the 
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tially  trained  people  who  have  received  training  from  colleges  in  business 
and  quantitative  analysis.  The  output  of  a  Cost  Analyst  is,  more  and 
more,  used  as  a  decision  tool  for  larger  (expensive)  decisions.  The 
quality  of  those  decisions  should  not  be  jeopardized  by  marginally 
qualified  estimations.  Training  with  industry  would  be  a  good  addition  to 
the  sources  of  training  used  in  the  questionnaire." 

"Summary  level  courses  should  be  established  to  provide  basic  information 
in  several  of  the  mathematical  courses  rather  than  indepth  courses.  In 
many  cases,  a  basic  knowledge  is  useful  and  necessary  but  time  to  take 
many  different  courses  is  not  available." 

"There  could  be  trail  exercises  (like  case  studies)  which  are  distributed 
periodically  to  the  various  dimensions  to  be  worked  and  sent  back  to  AFIT 
for  grading  and  critique.  This  could  be  done  on  a  voluntary  or  as 
requested  basis  or  as  a  requirement  for  new  troops.  You  could  even  have 
progressively  more  difficult  exercises." 

"The  greatest  experience/education  weakness  is  that  gov't  cost  analysts  do 
not  have  an  appreciation  for  the  risk  element  which  business  must  live 
with.  The  risk  factor  motivates  their  every  action  most  government 
analysis/decisions  ignore  the  risk/profit  relationship.  It  is  hard  to 
teach  this  subject  in  a  classroom  environment  and  it  cannot  be  learned 
in  government.  One  solution  may  be  to  hire  cost  analysts  with  more  pri¬ 
vate  industry  experience." 

"1.  Maintain  entry  level  to  those  having  Bachelor  Degree  in  Cost  Related 
Fields  of  Study,  i.e.  Bus.  Admin,  Indust.  Engineering,  Economics, 
Statistics.  2.  Continue  making  available  AFIT  courses  as  those  above  and 
AFIT  masters  in  Cost  Analysis,  DSMC  and  civilian  University  training.  3. 
Make  career  progression  available  to  military.  Do  not  penalize  for  narrow 
field  of  speciality." 

"The  current  training  program  is  extremely  good.  Once  the  basic  methods 
are  understood,  practice  on  "real"  programs  will  have  their  experience  and 
increase  their  capabilities.  The  biggest  aid  to  cost  analysis/estimating 
has  been  computer  programs  and  the  electronic  spread  sheet.  Our  people 
need  to  continue  their  training  in  this  area." 

"Get  the  people  capable  of  OJT  instruction  doing  the  instruction  rather 
than  the  many  personnel  tasks  they  are  assigned.  Also  don't  make  class 
attendance  a  course  for  cost  analyst  certification,  try  demonstrated 
performance." 

"The  basic  functions  of  a  cost  analyst  revolve  around: 

General  Air  Force  Operating  Procedure 

Statistics 

Accounting 

Computer  Programming  &  Operation 
Communications  (Oral  &  Written) 

F inance 
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"Promote  only  those  who  demonstrate  competence,  not  just  attend  the 
required  number  of  AFIT  courses." 

"Some  technical  training  (i.e.  Engineering,  Manufacturing)." 

"Establish  more  stringent  background  requirements  for  entrants  into  the 
field.  For  the  most  part,  the  technical  courses  presented  in  the  last 
section  (AFIT)  simply  do  not  suffice  in  providing  the  depth  of  knowledge 
desired.  Require  study  beyond  the  basics.  Any  worthwile  program  will 
also  contain  developmental  training  closely  allied  with  the  industry  where 
most  of  the  defense  contracting  is  undertaken.  This  can  be  accomplished 
by  close  attention  to  assignments." 

"Formal  DOD  classes  (Ex.  AFIT)  are  too  theoretical.  Little  application  to 
on  the  job  requirements.  We  need  more  practical  training." 

"Requirement  of  an  undergraduate  degree — especially  one  with  a  strong 
mathematical  &  statistial  background  (i.e.  business  administration,  opera¬ 
tions  research,  math,  or  engineering).  Continued  emphasis  on  upgrading 
skills  through  PCE  courses,  seminars,  symposiums,  etc.  An  additional 
suggestion  would  be  that  to  reach  the  journeyman  level,  each  cost  analyst 
must  have  acquired  a  certificate  of  professional  achievement  (ex-ICA  cer¬ 
tificate,  NES  certificate,  or  certificate  of  Professional  Designation  from 
AFIT)." 

"AF  should  recommend  analysts  continue  their  education  at  least  once  every 
year  both  to  add  knowledge  and  refresh  themselves.  Too  many  analysts  drop 
to  poor  quality  partially  because  they  let  their  education  lapse  or  are 
unwilling  to  travel  to  PCE.  One  reason  for  not  wanting  to  attend  many  PCE 
courses  is  their  length.  Any  course  beyond  2  weeks  is  unsatisfactory  for 
most  mid  to  older  analysts.  More  use  should  be  placed  in  university  cour¬ 
ses  in  the  local  area." 

"No  course  is  useless  for  a  cost  analyst.  They  should  not  only  be  con¬ 
cerned  ith  all  this  technical  schooling  but  should  also  join  in  opera¬ 
tional  type  schooling.  This  would  give  the  technocrat  a  broader 
understanding  of  the  role  of  the  DOD/AF  and  provide  him  more  of  an  opera¬ 
tional  viewpoint.  Give  cost  analysts  OJT  with  operational  divisions  to 
experience  how  it  really  is." 

"Only  hire  civilians  or  place  military  personnel  into  the  cost  analysis 
field  that  have  the  math  education  and  then  send  them  to  the  appropriate 
AFIT  classes  listed  above." 

"Cost  analysts  have  widely  varied  responsibilities  dependent  upon  the 
level  of  tasks.  Background  for  Wing/Base  level  analyst  is  less  demanding 
than  that  necessary  for  Air  Logistic  Centers  or  Air  Force  Command  levels. 

A  major  evolutionary  change  is  presently  underway  in  which  the  cost  ana¬ 
lyst,  who  was  qualified  to  audit  calucations  on  a  small  proposal  for  a 
piece  of  equipment,  must  have  the  background  to  develop  estimating  rela¬ 
tionships  from  scarity  data,  understand  private  industry  cost  methods  and, 
without  direction,  apply  the  appropriate  methods.  Since  it  is  not  prac¬ 
tical  to  train  personnel  completely  the  Air  Force  should  start  with  par- 
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"There  could  be  some  improvement  in  education  if  the  courses  and  instruc¬ 
tors  were  more  familiar  with  the  real  world-the  way  things  are  as  opposed 
to  how  they  should  be.  Also  some  quick  turnaround  techniques  would  be 
nice  since  alot  of  the  time  suspenses  are  very  short  and  the  techniques 
taught  at  AFIT  are  usually  time-consuming.  In  fact  the  best  way  to  become 
a  qualified  cost  analyst  is  to  take  the  classes  for  background  and  then  do 
estimates.  All  the  classes  in  the  world  can't  take  the  place  of 
experience." 

"You  can  train  all  the  people  in  the  world  to  be  good  technical  cost  ana¬ 
lysts.  But  a  good  functional  cost  analyst  can  do  the  job  with  very  little 
technical  training.  You  need  common  sense,  curiosity,  and  desire  to  be  a 
good  analyst." 

"Cross  training  in  all  Comptroller  functions." 

"General  Comments  on  approach:  All  of  my  cost  analysis  experience  has 
been  obtained  at  a  MAJCOM  HQ  dealing  primarily  with  D&S  costs  investment 
decisions,  and  ad  hoc  managerial  questions  or  decisions;  my  answers  are 
couched  in  that  context.  Any  of  the  above  areas  of  knowledge  or  skills 
can  be  useful  to  a  cost  analyst.  However,  it  is  not  necessary  -  and 
perhaps  not  even  desirable  -  that  a  given  analyst  possesses  all  of  them. 

It  is  sufficient  -  and  perhaps  best  -  that  the  skills  be  available  in  the 
office  though  several  analyst,  e.g.  an  economist,  and  industrial  engineer, 
an  accountant,  etc.  Also,  although  not  necessarily  the  "best"  way,  much 
can  be  learned  through  OJT." 

"Provide  training  relevant  to  the  individuals  job.  All  cost  analysts  do 
not  require  the  same  training.  There  are  many  cost  analysts  who  require 
greater  knowledge  of  the  logistics  processes,  research  methods,  and  data 
sources.  An  emphasis  on  practical  application  rather  than  "theory"  would 
also  improve  the  competence  of  Air  Force  cost  analysts." 

"(1)  An  undergraduate  degree  in  one  of  the  quantitative  disciplines 
(Accounting,  Economics,  Statistics,  Mathematics)  -  required.  (2)  A  gra¬ 
duate  degree  in  Business  Administration  -  preferred.  (3)  For  those  not 
having  (1)  and  (2)  attendance  of  as  many  of  the  AFIT  QMT  series  courses  as 
possible,  or  (4)  Taking  selected  courses  in  quantitative  methods  at  local 
colleges  and  universities." 

"AFLC  may  be  a  typical.  Most  of  the  more  advanced  mathematical  &  cost 
estimating  techniques  are  of  little  value.  The  non-availability  of  base 
data  makes  the  use  of  rudimentary  techniques  difficult  to  achieve  at 
times.  This  i3  compounded  by  the  constant  use  of  "political"  decisions 
versus  "economic." 

"I  believe  every  AF  analyst  should  be  exposed  to  a  variety  of  methods 
which  have  been  used  to  estimate  real  programs  rather  than  theory.  For 
example,  several  ICA  briefings  could  be  presented  during  the  AFIT  classes 
to  give  the  analysts  a  feel  for  real  world  applications  of  the  theory  they 
are  learning.  Also,  in  our  hiring  process  we  should  begin  to  emphasize 
more  math  and  technical  (science  and  engineering)  education  in  the  posi¬ 
tion  description." 
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Appendix  B:  Written  Comments 


The  following  are  comments  to  the  open-ended  question  at  the  end  of  the 
questionnaire.  No  effort  was  made  to  correct  grammar,  spelling  or 
expression.  The  question  was  written: 


As  a  final  task,  briefly  describe  how  you  feel  the  educational  competence 
(that  is,  adequate  knowledge  for  job  performance)  can  be  improved  for  Air 
Force  cost  analysts. 


Comments : 

"New  analysts  should  be  enrolled  in  a  formal  training  program  that 
requires  a  mandatory  job  rotation  an  organization." 

"The  courses  currently  offered  are  adequate  to  cover  all  aspects  of  need 
training.  Problems  center  around  the  time  needed  for  full  professional 
development  versus  the  rapidly  increasing  workload  and  the  low  manning 
levels  of  most  CA  organizations.  I  also  believe  that  many  of  the  per¬ 
ceived  educational  competence  problems  are  more  imagined  than  real  and 
stem  in  part  from  desires  to  obtain  professional  job  series  recognition 
rather  than  what  is  really  required  to  do  most  of  the  day-to-day  CA  busi¬ 
ness.  *  this  can  vary  significantly  depending  on  assigned  respon¬ 
sibilities  of  the  cost  analyst." 

"By  management  action.  Hire  people  who  have  appropriate  skills  continuing 
education,  maybe  one  course  a  year." 

"Have  recognized  separate  career  field  for  federal  cost  analysts.  Have 
active  professional  organization  that  speaks  for  cost  analysis  (NES  and 
ICA  haven't  done  enough  yet).  Universities  won't  offer  cost  analysis 
until  it  is  a  better  defined  discipline." 

"Through  costing  techniques." 

"I  believe  there  are  basically  two  groups  of  people  who  are  not  qualified 
in  cost  analysis  due  to  lack  of  education.  Military  (blue  suit)  personnel 
typically  are  not  required  to  complete  the  "cost  education"  course  work 
that  civilians  must  accomplish  to  be  promotable.  As  such,  military  cost 
analysts  tend  to  be  more  managers  than  workers.  Also,  (the  second  group) 
Senior  cost  analysts  tend  not  to  be  current  on  the  latest  techniques, 
research  etc,  due  to  the  dated  course  that  they  cover  up  the  ranks  with 
(no  computer  experience).  I  attend  one  course  a  year  to  keep  current." 

"Improved  by  broadening  people  in  the  area  to  understand  how  their  role 
fits  into  the  overall  Air  Force  mission.  Not  just  R  &  D  and  Procurement 
but  al3o  operations  once  the  system  is  fielded." 

"By  accepting  only  those  officers,  Airmen,  and  civilians  who  have  integral 
calculus  or  above." 

"More  emphasis  on  training  of  personnel  to  real  world  and  less  time  on 
theories . " 
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1  ... 

, ..  critical 

OJT  ... 

. ..  on-the-job  training 

2  ... 

. ..  extremely 

PCE  . . . 

. ..  professional  continuing 

education  program 

3  ... 

. ..  useful 

UG 

. ..  undergraduate  level 

4  . . « 

. ..  of  little  use 

GRAD  ... 

. ..  graduate  level 

5  ... 

. ..  useless 

(34) 

(QMT  540) 

ADVANCED  PRICING  METHODS:  regression  analysis;  sources  and  uses  of  data; 

and  analysis  of  contractor  financial  data 

1  2  3  4  5  OJT  PCE  UG  GRAD 

(35) 

(SYS  362) 

COST  SCHEDULE  CONTROL  SYSTEM  CRITERIA:  maintenance,  surveillance,  and 

evaluation  of  comprehensive  mana¬ 
gement  systems  to  include  skills 
necessary  for  demonstration/ 
validation  examinations  at  DOD 
contractors 

1  2  3  4  5  OJT  PCE  UG  GRAD 

As  a  final  task,  briefly  describe  how  you  feel  the  educational  competence 

(that  is,  adequate  knowledge  for  job  performance)  can  be  improved  for  Air 

Force  cost  analysts. 


critical 

extremely  useful 

OUT  ... 

PCE  . . , 

on-the-job  training 

, ..  professional  continuing 
education  program 

useful 

UG 

, ..  undergraduate  level 

of  little  use 

GRAD  • • < 

, graduate  level 

useless 

(29) 

(QMT  345) 

INTRODUCTORY  QUANTITATIVE 
ANALYSIS: 


probability;  descriptive  and  inferential  sta¬ 
tistics;  sampling;  unit  and  cumulative 
learning  curves;  index  numbers;  time  series 
forecasting;  two  variable  linear  regression 


1  2  3  4  5 


(30) 

(QMT  550) 

ADVANCED  QUANTITATIVE  METHODS: 


introduction  to  matrices,  non-linear  rela¬ 
tionships;  logarithms;  curvilinear  func¬ 
tions;  linear  and  non-linear  regression; 
of  two  variable  and  multivariable  rela¬ 
tionships;  net  scatter  diagrams; 


1  2  3  4  5 

(31) 

(QMT  551) 

ADVANCED  COST  AND  ECONOMIC 
ANALYSIS: 


advanced  regression  techniques;  treatment  of 
uncertainty  in  cost  analysis;  economic  analy¬ 
sis;  s imulataneous  estimation  techniques;  use 
of  models  and  estimation  of  operating  and 
support  costs 


1  2  3  4  5 

(32) 

(QMT  180) 

LEARNING  CURVE  ANALYSIS: 


1  2  3  4  5 


thorough  understanding  of  learning  curve  theory 
and  application 


(33) 

(QMT  353) 

INTRODUCTION  TO  LIFE  CYCLE 
COST  MANAGEMENT: 


life  cycle  cost  management  environment,  cost 
drivers,  estimating  techniques,  and 
contracting  aspects  of  life  cycle  costs 


1  ... 

2  ... 

...  critical 

. ..  extremely  useful 

OJT 

PCE  . . . 

. ..  on-the-job  training 

. ..  professional  continuing 
education  program 

3  ... 

. ..  useful 

UG 

. ..  undergraduate  level 

4  ... 

...  of  little  use 

GRAD  ... 

. ..  graduate  level 

5  ... 

. ..  useless 

OPERATIONS  RESEARCH 

(26) 

QUANTITATIVE  DECISION 

MAKING:  linear  programming;  inventory  theory;  queuing 

theory;  and  decision  theory 

1  2  3  4  5  OJT  PCE  UG  GRAD 


OTHER  SUBJECT  AREAS 


(27) 

SEMINAR  IN  COST  ANALYSIS:  role  of  the  cost  analyst;  code  of  ethics;  life 

cycle  cost  estimation  and  management;  effects 
of  discounting;  risk  and  uncertainty;  economic 
analysis;  independent  cost  estimates;  documen¬ 
tation  management  of  the  cost  analysis  function 

1  2  3  4  5  OJT  PCE  UG  GRAD 


The  QMT  courses  that  follow  are  offered  by  the  Air  Force  Institute  of 
Technology  (AFIT)  for  professional  development _ 


(28) 

(QMT  170) 

PRINCIPLES  OF  CONTRACT  PRICING:  tools  and  techniques  for  pricing;  methods 

for  analyzing  direct  and  indirect  costs; 
arithmetic  and  log-log  graphing;  unit 
learning  curve  formulation 

1  2  3  4  5  OJT  PCE  UG  GRAD 
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REGRESSION  I: 


(22) 

REGRESSION  II: 


regression  analysis  to  include  problems  involving  both  two 
variable  and  multivariable  linear  regression  and  the 
assumptions  of  each;  examination  of  casualty  and  indicator 
variables 

5  OJT  PCE  UG  GRAD 


regression  analysis  to  include  considerations  of  specifi¬ 
cation,  identification,  and  collinearity ;  translation  of 
data  into  information  for  decisions 


1  2  3  4  5 


OJT  PCE  UG  GRAD 


QUANTITATIVE  ANALYSIS:  development  of  engineering  estimates;  learning 

curve  theory;  index  numbers;  time  series  fore¬ 
casting;  and  technological  forecasting 


1  2  3  4  5 


OJT  PCE  UG  GRAD 


************************************************  **************************** 
COMMUNICATION 


(24) 

TECHNICAL  COMMUNICATIONS: 


writing  and  consideration  of  related  problems 
of  grammar,  syntax,  and  mechanics;  written  and 
oral  assignments  to  demonstrate  competence  in 
organizing  and  presenting  technical  infor- 
uiat  ion. 


(25) 

RESEARCH  METHODS: 


data  gathering;  information  measurement  scales;  classi¬ 
fication  of  variables;  validity  and  reliability; 
research  populations  and  sampling;  and  designs  to  test 
research  hypotheses  and  answer  research  questions 
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1  ... 

. ..  critical 

OJT 

2  ... 

. ..  extremely  useful 

PCE 

education  program 

3  ... 

. ..  useful 

UG 

4  ... 

...  of  little  use 

GRAD 

.  graduate  level 

5  ... 

. ..  useless 

************************************************************************** 
COMPUTER  SCIENCE 


(16) 

FORTRAN  PROGRAMMING:  fundamental  principles  of  computer  programming  basic 

elements  of  structured  FORTRAN 

1  2  3  4  5  OJT  PCE  UG  GRAD 

(17) 

BASIC  PROGRAMMING:  using  "basic"  language  to  model  and  analyze  typical 

management  problems;  common  programming  using  struc¬ 
tured  BASIC. 

1  2  3  4  5  OJT  PCE  UG  GRAD 

(18) 

COBOL  PROGRAMMING:  introduction  to  COBOL;  common  applications  using  the 

structured  COBOL  language. 

1  2  3  4  5  OJT  PCE  UG  GRAD 

************************************************************************* 

STATISTICS 

-l.  l-.i_.J_  >■  ■  «  I  .  ■  1  I.  I.  I  i  i-  ».  I-  .  I  ■ .  1..1.  I.  1  -1-  I.  I.  1--I.  t  I  I.  i  1..  I .  ,  1_ 

nw  A  »  WW  W  WWWmmWwWw  PpWrtW  W  n  A  Wn  WWa  FpWmWFTw  W  W  K  oWrt  cf  rlw  AS  jif 

(19) 

INTRODUCTORY  STATISTICS:  application  of  basic  probability  theory;  basic 

statistical  tests;  hypothesis  tests  and  for¬ 
mulations;  data  manipulation  and  evaluation  for 
statistical  decisions. 

1  2  3  4  5  OJT  PCE  UG  GRAD 

(20) 

MANAGERIAL  STATISTICS:  probability  theory  for  advanced  statistics;  tech¬ 
niques  for  summarizing  and  describing  data;  use  of 
statistics  for  communicating  information 

1  2  3  4  5  OJT  PCE  UG  GRAD 


critical 

OJT  .., 

. ..  on-the-job  training 

extremely  useful 

useful 

PCE  . . , 

. ..  professional  continuing 
education  program 

of  little  use 

UG 

. ..  undergraduate  level 

useless 

GRAD  ... 

. ..  graduate  level 

(10) 

MANAGEMENT  INFORMATION 

SYSTEMS:  development  and  management  of  information 

systems;  computer  applications  to  develop  an 
accounting  data  base  useful  for  management  deci¬ 
sion  making. 


PRODUCTION  MANAGEMENT:  robotics;  material  requirements  planning;  work 

measurement;  producibility;  quality  assurance; 
computer  aided  design;  computer  process  moni¬ 
toring;  group  technology;  computer  aided  manufac¬ 
turing. 


DEFENSE  PRODUCTION  MANAGEMENT:  weapon  system  manufacturing  technology; 

manufacturing  management  reviews;  various 
defense  production  management  regulations; 
defense  technology  modernization  efforts. 


** ************************************************************ *  *  ********* 
MATHEMATICS 

********  k**  it  ************************************************************* 

(13) 

MATRICES  AND  LINEAR  ALGEBRA:  fundamental  knowledge  of  matricies;  systems 

of  equations;  determinants;  vector  spaces; 


CALCULUS:  differential  and  integral  calculus 
12345  OJT  PCE  UG  GRAD 

(15) 

LINEAR  MATHEMATICAL  MODELS:  linear  equations;  inequalities;  convex  sets; 

matrix  algebra;  linear  programming;  duality; 
applications  of  linear  mathematical  models 
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1 


.  critical 

.  extremely  useful 

OJT 

PCE  . . , 

. ..  on-the-job  training 

. ..  professional  continuing 
education  program 

.  useful 

UG 

. ..  undergraduate  level 

.  of  little  use 

GRAD  ... 

. ..  graduate  level 

.  useless 

MANAGEMENT 


(5) 

TECHNOLOGICAL  FORECASTING:  state-of-the-art  techniques  for  technological 

forecasting  in  R&D  and  other  related  areas; 
the  Delphi  Method;  growth  curves;  various 
other  relevant  mathematical  models. 

1  2  3  4  5  OJT  PCE  UG  GRAD 

(6) 

FEDERAL  FINANCIAL  MGT:  fiscal  policy;  the  federal  unified  budget;  the 

congress ional  budget  process;  the  appropriation 
process;  fund  control  requirements;  the  planning, 
programming,  and  budgeting  system. 

OJT  PCE  UG  GRAD 


financial  management  of  business  units  with  emphasis 
on  finance  organization  structure;  collecting  and 
using  financial  data;  judging  profitability, 
liquidity,  and  sources  of  capital;  internal  financial 
operations . 

OJT  PCE  UG  GRAD 


1  2  3  4  5 

(7) 

MANAGERIAL  FINANCE: 


1  2  3  4  5 


(8) 

MANAGEMENT  AND  BEHAVIOR 

IN  ORGANIZATIONS:  classical  and  contemporary  theories  dealing  with 

the  management  of  complex  organizations;  systems 
approach  to  management  and  value  of  the  human 
resource . 


1  2  3  4  5  OJT  PCE  UG  GRAD 

(9) 

R&D  MANAGEMENT:  research  and  development  management  as  it  is  done  in 

Air  Force;  horizontal  integration  in  complex  organiza¬ 
tions  and  practical  applications  of  management  theory 
to  Air  Force  problems  and  roles. 

12345  OJT  PCE  UG  GRAD 


RATING  GUIDE 


1  . . 

...  critical 

OJT  . . . 

. ..  on-the-job  training 

2  . . 

...  extremely  useful 

PCE  . .  , 

. ..  professional  continuing 

education  program 

3  .. 

...  useful 

UG 

. ..  undergraduate  level 

4 

. . .  of  little  use 

GRAD  ... 

. ..  graduate  level 

5 

...  useless 

ACCOUNTING  AND  ECONOMICS 

*************************************************************************** 

(1) 

FINANCIAL  AND  COST  ACCOUNTING:  inventory  costing;  depreciation  methods 

analysis  of  financial  statements;  direct 
and  indirect  costs;  overhead  allocation; 
job  and  process  costing;  cost-volume- 
profit  relationships;  variance  analysis 
and  standard  costs. 

1  2  3  4  5  OJT  PCE  UG  GRAD 

(2) 

MANAGERIAL  ACCOUNTING:  cost  concepts;  cost  behavior  and  costing  tech¬ 
niques;  uses  and  limitation  of  cost  data  in 
planning,  controlling,  and  nonroutine  decision 
making. 

1  2  3  4  5  OJT  PCE  UG  GRAD 

(3) 

MICROECONOMICS:  cardinal  utility  measurement;  income  and  substitution 

effects;  derivation  of  demand  curves;  production  and  cost 
functions;  industry  structures;  resource  allocation. 

1  2  3  4  5  OJT  PCE  UG  GRAD 

(4) 

MACROECONOMICS:  optimization  subject  to  constraints;  risk  analysis;  tech¬ 

niques  of  demand  estimation;  empirical  cost  estimation 
and  analysis;  pricing  practices;  capital  budgeting. 

1  2  3  4  5  OJT  PCE  UG  GRAD 
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AIR  FORCE  COST  ANALYSTS  EDUCATIONAL  REQUIREMENTS  SURVEY 


PURPOSE: 

The  purpose  of  this  survey  is  to  solicit  opinions  from  experienced  Air 
Force  Cost  analysts  to  determine  the  educational  requirements 
necessary  to  establish  a  standard  educational  requirements  list  for 
judging  educational  competence  among  Air  Force  cost  analysts. 

Please  circle  the  appropriate  background  information  below: 

Major  Command:  AFSC  AFLC  MAC  SAC  TAC  ATC  OTHER:  _ 

Experience  (in  terms  of  years  in  the  cost  analysis  field): 


a. 

1 

to 

3 

years 

c.  7  to  10  years 

b. 

4 

to 

6 

years 

d.  11  or  more  years 

TASK: 

Rate  the  following  courses  from  1  to  5  as  they  apply  to  the  job  of  an 
Air  Force  cost  analyst.  Identify  the  appropriate  educational  level  or 
levels  for  acquiring  knowledge  in  the  courses  rated.  There  are  nine 
categories  with  several  courses  under  each  category.  Each  course  is 
followed  by  a  brief  description. 

(CIRCLE  YOUR  CHOICES  BASED  ON  THE  FOLLOWING  GUIDE): 


1  .  Knowledge  of  this  area  is  critical 

2  .  Knowledge  of  this  area  is  extremely  useful 

3  .  Knowledge  of  this  are  is  useful 

4  .  Knowledge  of  this  area  is  of  little  use 

5  .  Knowledge  of  this  area  is  useless 


OJT  .  Knowledge  should  be  acquired  through  on-the-job  training 

PCE  .  Knowledge  should  be  acquired  through  a  professional  con¬ 

tinuing  education  (PCE)  program. 

UG  .  Knowledge  should  be  acquired  at  undergraduate  level 

GRAD  .  Knowledge  should  be  acquired  at  graduate  level 
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guiding  hand  of  an  expert  estimator.  Thus  _i_f  a  good  cost  estimating 
course  were  to  be  developed,  the  instructors  would  have  to  br  seasoned 
estimators  to  be  effective." 

"Need  to  train  individual  specifically  as  cost  analysis.  Curren  AFIT 
program  is  definately  needed.  Needs  to  be  expanded  to  more  civilians  and 
more  trainers  need  to  be  trained  as  cost  analysts  as  opposed  to  financial 
specialists  by  means  of  continuing  education  courses." 

"The  AFIT  courses  listed  should  be  placed  in  a  linear  program,  i.e., 
prerequisites  set  for  firm,  controlled  progression  to  the  next  course 
level,  where  entrants  must  have  had,  as  a  minimum,  mathmatics  thru  basic 
integral  calculus  (usually  6  semester  hours)  and  introductory 
statistics/probability  theory  and  demonstrated  an  understanding  of  the 
material  by  maintaining  at  least  a  3.0  GPA,  on  a  4  pt  scale,  for  the  math 
&  stat  courses  taken.  Only  in  this  fashion  can  the  prospective  Jr.  Cost 
Analyst  get  the  maximum  benefit  of  the  AFIT  QMT  program.  Too  many  stu¬ 
dents  have  been  placed  in  the  AFIT  CA  program  w/o  enough  background  to 
effectively  concentrate  on  the  techniques  being  taught,  i.e.,  learning 
basics  at  the  same  time  detracts  from  learning-the  CA  material  being 
stressed — I  know,  that's  how  I  had  to  do  it  in  '74!!  (I've  picked  up  9 
hours  of  CAL  and  6  hours  or  stats  since — but  I  needed  them  in  '74!!)." 

"Bring  the  courses  closer  to  the  students'  location.  Make  the  courses 
shorter,  more  condensed  ( 1  or  2  months  is  too  long  to  be  away  from  the  job 
for  most  organizations)  Provide  more  courses  as  non-resident,  correspon¬ 
dence  courses.  Structure  the  course  so  that  student  could  attend  classes 
on  Saturday  and  Sunday,  study  on  Monday  through  Friday  then  return  again 
on  Saturday  and  Sunday  for  as  long  as  necessary  to  complete  course 
material.  Educational  competence  cannot  be  improved  unless  you  can  get 
the  students  into  the  class  room." 

"Cost  analysis  shop  ust  assure  their  people  are  properly  trained. 

Because  of  the  apparent  lack  of  understanding  by  the  Staffing  and 
Classification  people  we  will  continue  to  receive  untrained/underskilled 
personnel  for  our  vacant  c^st  positions.  We  have  found  through  experience 
at  SA-ALC  anyone  coming  into  our  shop  whether  a  GS-5  trainee  or  a  GS-12 
Accountant  or  Logistician  all  require  approximately  2  years  of  PCE,  UG, 
and  OJT  before  becoming  competent  Cost  Analysts.  Because  of  this 
problem  we  prefer  to  hire  GS-5  trainees  and  train  our  own  cost  analysts. 
This  is  a  problem  due  to  the  requirement  for  stepping  stone  positions 
which  we  lose  to  other  programs.  For  instance  we  recently  lost  our  GS-5-9 
3 1 ot  to  the  AFIT  Cost  Analysis  Officer  program.  Therefore  we  will  receive 
an  educated  officer  who  will  have  no  experience.  We  will  train  through 
OJT  for  several  years  then  lose  him  to  PCS.  In  the  mean  time  if  we  lose  a 
journeyman  civilian  we  will  have  to  drop  the  grade  and  train  another 
thereby  losing  a  large  portion  of  our  work  force." 

"We  need  more  communication  between  AFIT  instructors  and  the  supervisors 
of  the  people  who  take  AFIT  short  courses  so  that  course  material  can  be 
better  fitted  to  the  needs  of  the  students." 

"Establishment  of  formal  curriculum  at  universities  perhaps  a  bachelor 
degree  program." 
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"With  all  of  the  constraints  and  outside  influences  there  is  nothing  that 
can  be  done  that  will  improve  Air  Force  cost  analysts  but  then  again  we 
perform  extremely  well  given  the  above." 

"I  think  a  college  background  is  the  most  important  because  the  person 
puts  in  enough  time  and  is  graded  according  to  performance.  This  power¬ 
fully  reinforces  learning  much  more  effectively  than  PCE.  College  would 
cover  statistics  and  business  primarily.  Hopefully  some  science  too,  so 
problem  solving  approaches  are  taught.  A  strong  mathematics  background 
helps  in  analytical  ability  and  helps  understand  numbers.  The  background 
described  prepares  a  person  for  looking  at  the  reasonableness  of  a  problem 

solution.  With  this  background  existing  courses  at  AFIT  help  fine  tune 

skills  so  they  are  specific  to  the  requirements  of  cost  analysis." 

"The  Comptroller  training  program  at  ASD  is  super.  Persons  who  accomplish 
the  required  OJT  and  AFIT  courses  should  be  well  equipped  to  do  cost  ana¬ 
lysis  work.  There  are,  however,  other  qualities  which  I  believe  are 
essential  to  a  good  cost  analyst — agress iveness ,  and  just  a  tad  obnoxious. 
Cost  analysts  frequently  operate  in  an  adversarial  role  and  without  these 
qualities  are  generally  ineffective.  Don't  believe  the  ASD  cost  analysis 
training  program  can/needs  to  be  improved." 

"Make  it  easier  to  get  slots  in  AFIT  courses.  Increase  course  offering  in 

quarter  1,  2  &  3  -  people  can't  be  spared  4th  quarter." 

"Making  completion  of  selected  QMT  courses  such  as  QMT  170,  180,  345,  550, 
&  551  mandatory  prior  to  awarding  of  the  fully  qualified  674X  AFSC." 

"Entry  level  cost  analyst  should  have  strong  math  backgrounds  before  going 
into  the  field." 

"The  AFIT  courses  in  cost  analysis  should  be  expanded  so  as  more  cost  ana¬ 
lysts  have  the  opportunity  to  attend.  In  addition.  Air  Force  AC  should 
require  each  command  to  establish  a  training  package  to  ensure  all  cost 
analysts  have  a  basic  core  of  formal  education  courses  or  are  offered  the 
opportunity  to  tak«?-  the  classes." 

"More  on-the-job  training  and  experience.  In  short,  getting  your  feet 
"wet"  early  on  and  staying  with  it." 

"Generally  speaking,  I  feel  Air  Force  cost  analysts  have  good  training  in 
academic  tools  such  as  regression  analysis  and  learning  curves.  This 
major  deficiency  is  lack  of  understanding  of  the  technical  aspects  of 
their  weapons  systems.  Cost  analysts'  capabilities  would  be  improved  by 
greater  understanding  of  technology,  manufacturing  processes,  etc." 

"Periodic,  mandatory  PCE  for  cost  analysts  on  a  prescribed  curriculum. 
These  courses  should  be  semiannual  &  only  excusable  due  to  personal 
hardships.  If  not,  then  "mission  priorities"  will  always  supercede  PCE. 
Futhermore,  the  PCE  curriculum  should  be  tailored  by  command  (AFSC  vs  MAC, 
SAC)  to  allow  the  education  to  directly  help  current  duties." 

"Insure  analysts  get  to  available  courses  on  time.  Don't  let  job  require¬ 
ments  prevent  analysts  from  taking  courses.  Require  price  training." 


"Individualize  the  AFIT  courses  more  to  suit  the  needs  of  cost  analysts. 
Then,  standardize  the  requirements  for  qualification  as  a  AFSC  6746.  I 
became  a  6746  without  a  s ingle  AFIT  course  and  with  minimum  OJT." 

"Lieutenants  starting  a  career  as  a  cost  analyst  should  be  given  the 
opportunity  to  work  in  various  SPOs  and  the  staff  like  the  civilians  are 
before  being  assigned  to  a  permanent  position." 

"Assure  that  all  cost  analysts  have,  as  a  minimum,  the  knowledge  contained 
in  QMT  170  and  QMT  345.  Depending  on  where  they  work  and  their  job 
assignments  additional  knowledge  can  be  required.  The  best  way  to  assure 
this  knowledge  is  completion  of  the  appropriate  AFIT  courses  (PCE  for  most 
analysts)." 

"Through  organizations  such  as  National  Estimating  Society,  ideas  and 
techniques  can  be  transferred  so  that  we  do  not  become  over-specialized." 

"Introduce  people  into  the  field  with  stronger  math  backgrounds." 

"Better  communicative  skills  -  writing,  speaking,  briefing,  listening  (in 
laymen's  terms).  More  "Big  Picture"  orientation;  e.g.;  don't  get  lost  in 
minutes  when  rough  order  of  magnitude  is  all  that's  required.  Experience 
in  real,  base  level  world  vice  full  career  in  costing  at  SPO  or  MAJCOM 
levels ." 

"Develop  a  training  program  for  all  cost  analysts.  Program  should  be 
required  to  upgrade  to  6746.  In  addition,  program  should  include  basics 
required  to  perform  cost  analysis  work  both  in  acquisition  and  operational 
commands." 

"The  'Certified  Cost  Analysts'  and  the  'Professional  Designation'  are  a 
step  in  the  right  direction.  A  combination  of  experience  and  training 
should  allow  these  people  to  be  recognized  as  leaders  in  their  field. 
Additional  career  board  type  credit  should  be  given  to  those  people  who 
have  gained  the  two  certifications." 

"1.  Rotational  assignments  of  instructors  to  "the  real  world"  2.  On¬ 
site  instruction  courses  (warm  bodies,  not  teleteach)  3.  Training  plans 
that  are  adhered  too-not  just  squares  to  check  when  convenient." 

"Each  analyst  in  the  field  have  specific  needs  related  to  the  function 
being  performed  at  that  time  (R&D/Production  phone)  and  unique  to  the 
organization  (space,  armament,  etc).  The  field  organizations  should  pre¬ 
pare  a  planning  document  to  tailor  training  requirements  for  con¬ 
sideration  by  AFIT  to  meet  those  needs.  In  substance,  the  courses  and 
programs  in  place  are  adequate  to  enhance  educational  competance. 

However,  many  are  attending  classes  that  do  not  relate  to  the  criteria 
stated  above  and  many  are  denied  the  class  who  do.  Solution  is 
strengthening  the  justification  process  for  students  to  attend  the  scarce 
course  availability." 

"AFIT  needs  to  develop  videotape  cassettes  of  their  QMT  courses  in  cost 
analysis.  This  would  provide  the  opportunity  for  analysts,  who  find  it 
difficult  to  spend  4-6  weeks  away  from  their  job,  to  obtain  the  basic 
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understanding  of  cost  analysis  techniques.  Videotape  cassettes  should  be 
set  up  to  cover  several  of  the  various  analysis  techniques.  For  example, 
one  could  be  set  up  for  linear  and  non-linear  regression  models;  another 
cassette  could  include  estimating  and  forecasting  techniques,  etc,  etc. 
These  videotape  cassettes  would  not  replace  the  QMT  courses,  but  would 
upgrade  and  enhance  analyst  skills.  This  is  desperately  needed  in  the 
field.  AFIT  needs  to  restructure  their  courses  (i.e.  more  two  week  cour¬ 
ses  rather  than  4-6  week  courses)  and  more  on-the-road  training  at 
MAJCOM/HQ  AF  locations  rather  than  at  Wr ight-Patterson. " 

"Air  Force  cost  analysts  basically  work  with  contractor  furnished  infor¬ 
mation  within  AFSC.  The  best  way  to  do  a  good  job  is  to  understand  how  a 
contractor  manages  a  contract.  Learning  his  managment  system,  from 
planning  to  analysis  and  corrective  actions,  is  the  best  way  to  ensure 
thorough  and  proper  attention  to  the  real  problems.  (AFSC  Biased)  this 
can  be  accomplished  by  participating  in  C/SCSC  review  activity  and 
learning  the  inner  workings  of  companies  who  submit  data  for  analysis." 

"1.  Affiliation  and  participation  in  professional  organizations,  i.e., 
Institute  for  cost  analysis,  American  Society  of  Military  Comptrollers. 

2.  Greater  exposure  to  the  qualitative  aspects  affecting  a  quantitative 
art!  Leadership,  motivation,  and  individual  will  greatly  impact  produc¬ 
tivity  and  cost.  3.  Practice!!!  Homework!!!  Hard,  often  ridiculed, 
work." 

"Continue  to  emphasize  case  type  learning  methods  in  AFIT  short  courses. 
Cases  should  be  relevant  to  "real  world"  situations.  This  real  world 
aspect  has  been  lacking  in  previous  courses  time  attended.  There  is  a 
tremendous  need  to  improve  amount  of  computer  applications  for  cost  ana¬ 
lyst's  use.  Lack  of  experienced  analysts  makes  the  computer  an  invaluable 
tool.  Often  times  programs  used  at  AFIT  are  unavailable  for  use  back  in 
the  field." 

"After  being  assigned  as  a  cost  analyst  within  Systems  Command  and 
Communications  Command,  I've  seen  vastly  different  aspects  of  this  pro¬ 
fession.  The  educational  requirements  for  one  in  the  acquisition  process 
are  altogether  different  from  operations  and  maintenance.  A  fully 
qualified  cost  analyst  is  not  really  "fully  qualified"  when  moving  between 
commands  because  of  the  differences  in  the  nature  of  the  roles/missions  of 
each  command.  I  think  there  should  be  some  core  courses  one  should  take 
at  the  entry  level  that  provides  a  good  foundation  to  build  on.  Then, 
each  command  should  establish  a  series  of  courses  that  would  be  at  the 
journeyman  level  and  somewhat  command  and  mission  oriented.  These  courses 
would  be  broad-based  enough  to  accommodate  more  than  one  command,  yet  spe¬ 
cific  enough  in  individual  blocks  to  be  useful  when  performing  command 
related  activities." 

"The  training  program  is  more  than  adequate  in  the  class  environment. 

What  is  lacking  on  your  questionnaire  and  in  the  work  environment,  is  ade¬ 
quate  training/guidance  of  the  trainee  with  respect  to  OJT.  Students  come 
back  with  knowledge  of  new  techniques  and  methods,  but  do  not  receive  any 
immediate  opportunities  to  apply  them.  When  opportunities  arise,  super¬ 
visors  very  rarely  attempt  to  correlate  a  training  course  with  'real" 
applications." 
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"I.  Comprehensive  cost  training  program  at  entry  level  to  include  quan¬ 
titative  analysis  &  statistics  courses  at  AF1T  or  local  college.  2. 
Develop  courses  that: 

a.  identify  the  major  data  bases/systems  used  by  AF  costers. 

b.  give  an  overview  of  the  data  base's  content,  validity,  and  use. 

c.  structured  to  address  specific  types  of  costing,  e.g.,  R&D 
Logistics,  Production,  etc." 

"The  analyst  currently  all  taught  techniques  of  estimating,  but  see  them¬ 
selves  as  only  a  support  office.  Analyst  need  to  get  more  involved  with 
program  management.  Analysts  need  to  be  taught  Program  Mgt  skills  so  they 
can  understand  all  decisions  made  by  the  Program  Mgt  and  have  the  ability 
to  question  decisions  that  could  lead  to  program  cost  growth." 

"Air  Force  Cost  Analysts  perform  (at  least)  two  distinctly  different  types 
of  work.  Either  working  in  SPOs  or  on  the  Comptroller  Staff,  Cost 
Analysts  are  usually  Cost  Estimators  (Comptroller  ACCE)  or  C/SCSC  Analysts 
(Comptroller  ACCI).  In  SPOs,  C/SCSC  Analysts  often  are  used  as  Financial 
Managers  performing  C/SCSC  monitoring  functions,  where  C/SCSC,  Financial 
Management,  Communications,  and  the  bas ic  estimating,  statistics,  and  com¬ 
puter  courses  would  be  most  helpful.  On  staff  (e.g.  ACCI),  these  are  also 
very  important,  especially  the  C/SCSC,  communication,  and  computer 
training.  Training  in  DCAS/AFPRO,  and  DCAA  related  tasks  would  also  help. 
Cost  Estimators  are  much  more  in  need  of  the  more  quantitative  courses  for 
estimating  and  forecasting  for  specific  defense  programs.  Estimators  are 
often  trained  by  the  Comptroller  (often  ACCE)  and  "matrixed"  to  SPOs  on  a 
"long-term  temporary"  basis.  Often  (my  case  included),  6746s/Cost 
Analysts  receive  training  in  both  areas  and  then  uses  the  skills  of  only 
one  or  the  other.  College  graduates  with  more  quantitative  business 
backgrounds  should  be  directed  toward  Cost  Estimating.  Graduates  with 
more  managerial  and  systems  management  backgrounds  should  be  designated 
"C/SCSC  types."  In  both  cases,  training  could  be  concentrated  for  each 
group  in  the  respective,  specialized  Cost  Analysis  career  area,  with  less 
wasted  training  spent  on  officers  entering  the  Cost  Analyst  career 
"shredouts . " 

"After  the  Air  Force  cost  analyst  has  the  technical  competence  and  skills 
in  math,  statistics,  accounting,  etc.  he  still  faces  the  formidable  task 
of  being  able  to  apply  these  subjects  to  produce  timely  and  useful  cost 
estimates.  Success  from  this  point  is  largely  determined  by  how  well  one 
can  interpret  and  fulfill  the  desires  of  the  system  program  manager  (SPO) 
in  acquiring  and  managing  the  resources  (i.e.,  money)  to  acquire  a  given 
weapons/support  system(s).  Knowledge  of  the  budget  system,  contract 
pricing  principles,  staff  integration  and  planning  procedures  and  other 
non-quant itative  skills  combine  to  help  "sell"  the  cost  estimate  to  the 
organizations/offices  that  coordinate  on  SPO  projects.  Failure  to  get 
their  cooperation  can  cause  a  technically  accurate  cost  estimate  to  be 
rejected  by  the  program  director.  These  skills  (non-quantitative )  are 
usually  acquired  by  OJT  and  PCE  courses /seminars ,  along  with  trial-and- 
error  attempts." 

"Improve  computer  training  with  micro's  as  tools  for  the  analyst.  Use  of 
electronic  spread  sheets,  data  base  managers,  and  graphics  should  be 
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stressed.  Use  of  powerful  on  line  or  micro  resident  statistical  packages 
(SPSS)  should  be  emphasized.  Analysts  need  better  preparation  in  math. 

Not  so  much  Calculus  but  algebra,  and  statistics." 

"1.  Elimate  0-5  &  above  crossflows  from  rated  areas.  Recognize  that  this 
is  indeed  a  profession  &  not  just  anyone  can  step  in  and  manage/lead  a 
cost  shop  (nor  should  someone  with  no  comptroller  experience  be  expected 
to  do  well)  1  would  rather  see  0-6  positions  filled  with  qualified, 
experienced  captains  than  unqualified  individuals.  This  would  eliminate 
the  bulk  of  my  headaches.  2)  Education  -  insist  that  as  a  minimum. 

674X's  have  -  9  credit  hrs  UG  Stats,  9  hrs  UG  Accounting,  including 
Managerial  Cost  Accounting  -  Matrix  Math  -  Calc  optional.  3)  How  about  a 
PCE/OJT  Course  that  gives  an  exhaustive  review  of  all  Air  Force  data  bases 
&  how  to  access  them.  4)  A  worker  level  cost  symposium  held  annually 
(worker  level)." 

"Increase  application  of  theoretical  techniques  to  current  situations  the 
student  can  identify  with.  Clearly  specify  whether  a  technique  is  being 
illustrated  as  appropriate  to  the  Comptroller  Analyst  or  the  Procurement 
Analyst  —  journeyman  analysts  know  which  is  being  illustrated  but 
trainees  only  get  confused  about  where  or  how  the  technique  should  be 
used! " 

"Having  had  the  good  fortune  of  gaining  an  inordinate  amount  of  experience 
over  the  last  16  years  by  participating  on  many  programs  as  the  ICA  or  ICS 
Team  Chief,  source  selection  cost  Panel  Team  Chief,  and/or  ESD  represen¬ 
tative  on  high  visibility  DOD  work  studies  for  future  together  with  and 
outside  of  DOD,  I  feel  the  most  important  tool  for  success  in  the  field  of 
cost  analysis  is  "On  The  Job  Training".  Formal  class  training  is  a  must 
however,  there  is  no  substitute  for  actual  hands  on  experience.  Two  of 
the  most  educational,  demanding,  diversified,  dynamic  and  important  types 
of  efforts  to  acquire  this  competence  are  Independent  cost 
analysis/studies  and  source  selections  which  I  would  highly  recommend  as 
"musts"  for  any  trainee  type  cost  analyst." 

"Cost  Analysis  courses  offered  by  AFIT  are  not  very  rebutable  to  the  real 
world.  Too  much  Procurement.  Too  much  emphasis  on  Non  High  Technology, 
instructors  need  current  real  world  experience.  Do  poor  job." 

"The  AFIT  courses  are  good:  Require  cost  analysts  to  take/attend  the  AFIT 
courses  provided  for  cost  &  pricing  analysts  instead  of  OJT  and  a  desire 
to  attend  these  courses.  Layout  rqmt  to  take  QMT  170,  353  &  180  in  first 
18  mos  in  order  to  be  upgraded  (6741  to  6746).  Then  take  the  remaining 
QMT/SYS  courses  in  the  following  18  mos." 

"The  cost  analyst  requires  a  well  rounded  background  in  financial  and  cost 
accounting,  statistics,  mathematic,  economic,  operations  research,  and 
managerial  techniques.  However,  even  with  their  background,  no  substitute 
exists  for  experience  gained  from  OJT." 

"More  emphasis  on  the  following: 

-  task  definition  and  planning. 

-  data  gathering  and  validation. 


-  documentation  of  methodology. 

-  presentation  of  results." 

"One  or  two  courses  dealing  in  statistical  sampling." 

"Encourage  analysts  to  take  college  level  courses  in  math  and  statistics 
as  part  of  career  development  plans." 

"Create  courses  that  assist  in  real  job  performance,  not  just  a  group  of 
nice  to  have  courses  that  people  use  to  fill  up  their  resume." 

"A  formal  OJT  program  should  be  a  requirement  at  each  location  where  cost 
analysts  work.  The  program  should  be  tailored  to  the  unique  products 
estimated  or  managed.  The  luxury  of  being  fully  trained  prior  to  "jumping 
into"  the  job  is  usually  unaffordable.  Therefore,  it  is  imperative  that 
the  cost  analyst  be  given  the  maximum  opportunity  to  have  the  knowledge 
and  tools  to  do  the  job.  Although  most  cost  analyst  work  involved  cost 
analysis,  training  &  education  should  cover  cost  estimating,  cost  analysis 
and  cost  performance  analysis." 

"A  basic  problem  is  application  of  the  knowledge  base  material.  That  is, 
once  educated  how  do  you  quickly  apply  the  learning?  A  possible  approach 
would  be  to  establish  tailored  training  courses  to  various  environments 
(e.g.  operating  command,  acquiring  command,  supporting  command,  special 
commands,  joint  commands  etc).  In  addition,  effort  should  be  given  to 
establishing  training  in  Business  Advisory  capabilities  and  other 
Business/Financial  Council  Activities." 

"1st  give  the  analyst  OJT  experience,  then  work  in  the  cost  and  pricing 
schools  at  AFIT.  After  3-4  years  a  Master  Degree  Program  in  Ops  Research 
or  Business  Administration.  The  practical  work  experience  must  go  hand  in 
hand  with  the  educational  training.  Learning  curve  theory  cannot  be  fully 
understood  and  appreciated  without  being  in  the  job  environment  (Pentagon 
or  ASD)  where  the  theory  is  being  used  daily." 

"Cost  analysis  education  must  be  tailored  to  each  individual  as  each  per¬ 
son's  background  will  vary  upon  entry  to  the  cost  analysis  field.  Courses 
must  be  available  to  satisfy  any  deficiencies.  Many  of  the  courses  listed 
in  the  attached  survey  are  recognized  as  being  part  of  a  graduate  or 
undergraduate  degree  curriculum.  Requirements  for  those  courses  should 
continue  to  be  fulfilled  at  those  levels  and  not  PCE  or  OJT.  PCE/QJT 
should  be  reserved  for  those  educational  requirements  tailored  to  the 
AF/DOD  environment.  Additionally,  PCE  is  appropriate  for  topics  where 
graduate/undergraduate  courses  are  not  available." 

"There  is  no  unified,  coherent  program  to  develop  cost  analysts'  educa¬ 
tion.  Analysts  as  a  group  come  from  many  disciplines  including  engi¬ 
neering,  mathematics,  business,  accounting,  statistics,  and  economics.  As 
a  result,  the  same  cost  analysis  problem  may  be  viewed  differently, 
depending  on  the  perspective  of  whoever  is  doing  the  viewing.  I  would 
like  to  see  a  more  basic,  strutured  approach  to  incorporate  the  way  the 
Air  Force  (DOD)  does  business.  I  recognize  the  difficulties  of  this  since 
67xx's  perform  estimating  and  analysis  as  well  as  many  27xx  duties  but, 
having  worked  in  all  three  areas,  research  methods,  statistics, 
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regression,  modeling,  learning  curves,  financial  analysis,  earned  value, 
and  Che  use  of  micro-computer  spreadsheets  are  most  important  -  especially 
when  taken  in  the  context  of  DOD  systems  acquisition  -  This  is  the 
perspective  lacking  in  any  civilian  education.  On  the  other  hand  I  don't 
see  many  (any)  analyst  using  calculus  on  an  every-day  basis.  The  AFIT 
short  courses  are  fine  but  too  limited  in  scope,  to  hard  to  get  quotas  for 
and  especially,  to  far  apart  in  time.  OJT  can  be  exceptionally 
frustrating  if  the  trainee  is  not  motivated.  Why  not  a  3-4  month  course 
consoldiating  data  research,  statistics,  line  regression,  modeling, 
learning  curves,  financial  analysis,  earned-value ,  micro-computer  spread¬ 
sheets  -  for  junior  analysts.  Create  a  giant  "integrating  problem"  using 
all  these  techniques  in  the  gov't  procurement  environment.  Also  refresher 
training  -  and  state-of-the  art-  updates  would  be  nice  periodically.  I'd 
be  happy  to  discuss  this  with  anyone  who  cares  to." 

"I  may  not  be  a  good  "sample  point"  for  you,  I  started  in  1960,  when  we 
were  overwhelmed  by  the  arrival  of  Frieden  calculators,  so  we  didn't  have 
to  run  out  learning  curves  by  hand.  But  what  I've  felt  is  missing  too 
often  is  the  understanding  of  cost  drivers.  Why  does  speed  cost  more? 

Can  it  ever  cost  less?  How  does  a  production  live  work?  What  price 
missile  range?  CEP?  Does  frequency  drive  avionics  cost?  Does  range? 

What  else?  I  think  people  need  math  techniques,  but  even  more  they  need 
to  understand  why  a  particular  technique  is  best  in  a  given  situation. 

Good  luck  on  your  survey.  Its  got  lots  of  good  uses.” 

"Note  (28),  (34)  &  (35)  are  really  narrowly  oriented  but,  they  are 
necessary  for  the  overall  education  &  training  of  the  cost  analyst.  We 
need  basic  courses  in  the  time  value  of  money  for  our  new  cost  analysts 
and  because  of  the  impact  of  interest  rates,  etc;  economics  courses  should 
also  be  programmed  early  on  in  an  analysts  career.  There  is  also  a  need 
for  more  operations  research  type  education  and  training  for  the  cost  ana¬ 
lyst.  In  the  area  of  OJT  &  cross  feed;  we  in  CA  need  more  basics  in  finan¬ 
cial  mgt  -  our  jobs  cross  many  functional  lines  in  gathering  financial, 
workload,  etc,  data  that  are  compiled,  merged,  purged,  etc  into  some  kind 
of  forecast  for  future  programs  and  more  education  &  training  in  financial 
mgt  would  be  beneficial.  Establish  a  realistic  career/profess ional  program 
guide  from  which  to  program  ones  progress  through  the  varied  cost  areas. 
Apparently,  ASD  is  seeking  more  operating  command  involvement  in  source 
selection,  life  cycle  costing,  D  &  S/M  costing,  etc.  They  (ASD)  need  to 
establish  some  criteria  that  the  operating  commands  can  follow  when 
selecting  analysts  to  support  ASD.  In  the  final  analysis  -  the  AF  needs 
to  develop  realistic  course  curr.  where  the  analyst  gains  basic  knowledge 
in  techniques,  tools,  etc  and  then  enforces  these  newly  turned  procedures 
etc.  through  a  firmly  established  OJT  or  trng  with  industry  program.  I 
personally  have  been  to  only  a  couple  of  the  educational  courses  and  I 
have  been  basically  forced  to  develop  my  own  talents  -  sometimes  at  great 
frustrations  and  excessive  time  seeking  a  starting  point.  It  would  be,  in 
my  opinion,  more  cost  effective  to  provide  the  education  and  a  solid  OJT 
or  intern  program  than  it  is  for  us  to  flounder  around  in  the 
semi-dark! ! ! " 

"I  am  the  HQ  AFSC  Director  of  Cost  Analysis  -  and  you  need  to  understand 
where  I'm  coming  from  to  put  my  response  in  context.  I  am  a  trained 
engineer  (MS-Astro)  with  a  lot  of  math  &  science  (BS).  I've  had  virtually 
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no  AFIT  PCE,  but  have  been  to  the  DSMC  long  course,  I've  served  as  an 
engineer,  test  monitor,  program  manger,  program  controller  (plans)  and 
Director  of  Budget  and  Director  of  Cost  Analysis  -  as  well  as  aide  &  IG 
time  -  all  in  AFSC  at  ASD,  ESD  &  HQ.  My  answers  are  in  recognition  of  a 
current  situtation  that  includes  -  severe  growth  of  quantitative  workload 

-  severe  growth  of  qualitative  degree  of 
difficulty  -  i.e.  estimates  very  early  in  the  evaluation  of  the  programs. 

-  intense  heat  from  everywhere  to  do 


better. 


-  a  workforce  that  is  largely  untrained 
(or-undertrained) ,  inexperienced,  and  undersized  to  the  task  at  hand. 

At  the  MACRO  level  -  the  only  way  we  are  going  to  get  the  job  done  over 
the  intermediate  term  (0  to  7  years)  is  to  train  (Read  OJT)  our  way  out  of 
our  current  predicament  -  while  taking  advantage  of  whatever  AFIT  has  to 
offer  to  the  maximum  extent  possible.  Realistically  it  can't  all  be  done 
that  way.  For  the  few  who  can  get  into  the  AFIT  residence  GCA  program, 
cover  as  much  PCE  material  as  you  can  without  skipping  over  other  impor¬ 
tant  content.  Use  PCE  for  those  not  in  GCA  and  to  fill  in  deficiencies  in 
<JG  education.  Take  some  PCE  on  the  road,  so  students  can  attend  class  4 
hrs  and  work  6  hrs.  Encourage  off  duty  education.  But  considering  the 
need,  in  the  new  term,  most  analysts  will  need  to  learn  most  of  what  they 
learn  via  OJT.  We  could  use  some  help  in  this  area  through  the  develop¬ 
ment  of  programmed  texts  and  computer  aided  instruction  packages  on  a 
variety  of  quantitative  analysis  and  acquisition  management  topics.  I 
ought  to  point  out  that  not  everyone  in  the  career  field  shares  my  views. 
Many  feel  that  formal  AFIT  coursework  is  the  only  way  to  educate  our  young 
analysts.  But  AFIT  does  not  have  the  resources  to  train/educate  everyone 
even  if  AFIT  could  accommodate  them.  Our  biggest  single  problem,  in  my 
opinion,  is  that  we  have  too  many  estimators  who  simply  don't  understand 
the  acquisition  process  -  at  either  the  program  level  or  the  DOD  level. 
Chalk  it  up  to  lack  of  experience.  Our  second  biggest  problem  is  that 
lack  of  technical  familiarity  with  the  programs  leads  to  poor  assumptions, 
groundrules,  risk  analyses,  etc.  Chalk  this  up  to  lack  of  technical 
training,  lack  of  experience,  a  workforce  spread  too  thin,  and  a  lack  of 
knowledge  of  problem  solving  techniques.  Our  last  big  problem  is  lack  of 
estimating  skill  -  caused  by  insufficient  education  &  training  of  the 
current  workforce,  a  long  pipeline,  and,  to  an  extent,  a  mismatch  between 
course  content  and  functional  needs  (mostly  PCE).  Do  not  get  the 
impression  I'm  panning  AFIT  -  I  am  not.  But  AFIT  cannot  possibly  make  my 
problems  go  away  by  itself.  The  Graduate  program  looks  good,  and  is 
getting  better.  That's  my  source  of  the  functional  leadership  of  the 
future,  and  I  can  afford  a  mix  of  theory,  breadth,  and  pragmatism  gained 
over  a  15  month  period.  But  it  also  only  gets  me  half  a  dozen  analysts  a 
year.  Some  of  the  PCE  courses  could  probably  use  some  revision.  With 
regard  to  your  survey  questions  and  the  preferred  source  of  the  education. 
The  preferred  location  for  much  is  always  UG,  followed  by  GRAD  (where  the 
level  of  instruction  justifies  inclusion  in  a  grad  program.) 


The  cost  analyst  should  have  all  but  the  first  discipline  introduced  to 
him  in  undergraduate  education.  If  the  cost  analyst  does  not  have  them, 
it  is  necessary  for  the  individual  to  obtain  them.  After  a  cost  analyst 
has  a  general  understanding  of  all  the  disciplines,  the  next  process 
involves  refining  that  understanding  through  on-the-job 
training/education.  To  improve  the  performance  of  cost  analysts,  the 
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comptroller/supervisor  should  ensure  that  all  cost  analysts  have  the  basic 
understanding  of  all  disciplines  before  they  are  allowed  to  assume  posi¬ 
tions  in  which  they  lack  the  capability  to  perform.  On-the-job  training 
and  government/professional  education  will  enhance  the  cost  analysts  per¬ 
formance  as  they  gain  experience." 

"1)  Improve  the  timing  of  course-work  to  coincide  with  opportunities  on 
the  job  for  application  of  course  material.  2)  Need  increased  emphasis  on 
estimating  R&D  and  on  data  gathering/stratifying. " 

"Analyze  the  current  taskings  of  a  cost  analyst  at  various  MAJCOMDs  and 
determine  the  commonality  and  differences  of  duties.  Also,  determine 
future  taskings  that  a  cost  analyst  will  be  expected  to  perform.  This 
assessment  of  tasking  totality  should  address,  as  a  minimum  the  tools 
(e.g.  computer  literacy)  that  are  required  to  effectively  carry-out  his  or 
her  duties.  Based  on  this  assessment  a  commonality  of  MAJCOMD  taskings 
should  be  used  to  identify  required  Air  Force  educational  requirement. 
Also,  MAJCOMDS  for  the  unique  cost  analysis  tasking  for  there  MAJCOMDs 
should  implement  provisions  to  ensure  educational  opportunities  are  in 
place  to  supplement  the  basic  (COMMON)  Air  Force  educational  program  for  a 
cost  analyst.  The  point  is  the  educational  program  must  have  applicabi¬ 
lity  to  the  job.  The  initial  step  is  to  perform  a  thorough  functional 
analysis  of  MAJCOMDs  cost  analysis  program." 

"1)  Training  must  be  made  available  when  the  organization  needs  it,  not 
when  it  can  be  scheduled.  In  many  cases  the  individual  comes  on  the  job, 
is  scheduled  for  training  2  years  in  the  future  (if  we  can  get  a  space) 
and  leaves  the  organization  2-3  years  later  without  receiving  any 
training.  2)  At  least  videotape  some  of  the  most  useful  classes  and  get 
them  to  the  field  this  is  certainly  better  than  nothing  and  the  long,  long 
wait  to  get  trainees  in  classes.  Sometimes  we  are  expected  to  do  the  work 
without  any  training." 

"Improvement  of  the  educational  competence  of  Air  Force  Analysts  can  be 
attained  only  when  management  at  all  levels  realize  the  importance  of 
having  qualified  and  well-trained  cost  analysts.  Management  must  then  be 
willing  to  insure  that  the  proper  mix  of  QJT,  education,  and  completion  of 
professional  AFIT-type  course  is  available  to  all  cost  analysts.  As  long 
as  management  looks  at  cost  analysts  as  something  less  than  professionals, 
educational  competence  will  always  stand  in  need  of  improvement." 

"By  system  enhancements.  Systems  should  be  programmed  to  flag  areas  of 
concern,  i.e.,  where  repair  costs  exceed  new  buy  costs  or  where  current 
year  price  exceed  previous  year  price  by  more  than  10%. " 

"Job  rotation  to  broaden  OJT.  Emphasis  (i.e.  to  be  eligible  for  promo¬ 
tion)  on  formal  courses." 


85 


tLi  10  tLIS.tC  LtvEL  by  81  TO  <)]5'fUtJ8G 

CuMKlUUNG  TOR.. 

SUBJI  10  SU3JJ5,5UBJEQ1 j  VALUE..  I  FIK  I  CA 


rm  •  W - •  r - - If-  .  y-  ■  .  -  - — - — — 1 — *  ■  ■  "W  I  ^  ■  ■  ■  »  '  rn  1  K  ■'  l1  ■  V 


•  v  *  V v '  "• ■>■  \ u '.  i.  '.v  ■■.  t-tt^ 


Appendix  C:  Computer  Cross-tabulations 


Jt  < 
◦  •- 
X  O 


<v 


o  • 
o 


o 


M) 

7 

M  MM 

7 

M 

M  M  M 

M 

M  MM  M 

7 

M 

MMM 

7 

M  M 

M 

M 

M 

MMM 

7 

** 

** 

M 

<n 

n 

J* 

o 

0 

0 

0 

O 

o  o  o 

m  m  o  r* 

0 

0 

0 

0 

o 

9  O 

o 

o 

ooo 

o 

o 

o 

o 

M  M 

Ui 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

•  • 

• 

•  •  • 

• 

• 

• 

• 

-i 

ooo 

0 

0 

0 

MOO 

OOO 

o  o 

o 

ooo 

o 

o 

o 

O 

uj 

o 

cr> 

M 

Ui 

«ar 

■9 

OWN 

o 

o  oo 

<w  a 

a 

ooo 

o 

o  o 

o 

o 

ooo 

o 

o 

o 

o 

CM  CM 

9 

•  •  • 

•  •  • 

%  •  • 

•  •  • 

•  • 

• 

•  mm 

• 

• 

• 

M  • 

—  VM 

O  mfM 

ooo 

a*  9  m 

ooo 

ooo 

OOO 

o  o 

o 

<s 

►“  « 

Lf» 

9 

—  -J 

-J 

n  .  .  i 

-J 

OJOB7 

M 

o  o  in 

MM?  o 

M 

0 

V 

1 

M 

O  9 

9 

M 

o 

o 

o 

o 

— 

•  •  • 

«■• 

•  •  • 

K>  •  •  • 

•  •  • 

•  • 

• 

•  •  • 

• 

• 

• 

m  • 

■a. 

n  ^  M 

r*>»  o  r-» 

-A  O  M 

LO  —  O 

o  m 

M 

=r  —  o 

o  o 

o 

9 

U* 

ru 

in  cm 

m  9  <\i 

cm 

m 

M 

V) 

3 

Ui 

X  -4  <M 

o 

0 

0 

0 

9  9  in  — 

iTI/»w© 

M 

0»N 

CM  O  9 

9 

9* 

-NN 

o 

o 

o 

o 

CM  O 

LU  -J 

•  •  • 

•  •  • 

•  •  • 

•  * 

m 

•  •  • 

• 

• 

• 

vn  • 

X  U* 

o  ©  o 

—  —  o 

m  —  o 

o 

M 

r*r*(M 

o 

o 

o 

in 

—  LM 

m  m 

CM 

in 

in 

M 

X  V) 

9  2> 

-1 

1  1  T  ||  1  1  1 

(J 

MO  » 

J>  in  r*  m 

M  O  ^  J 

9 

o  o 

O 

M  -OCT  7 

M 

o 

9 

<r  r*. 

•  •  • 

•  •  • 

—  •  •  • 

•  •  • 

•  • 

• 

•  •  • 

• 

m 

• 

1M  • 

n  o  — 

o  «  — 

9  —  0 

0  9  — 

OOO 

9  9  — 

o 

o 

— 

CM 

M 

—  *n  — 

m 

CM 

o 

M 

X 

M 

CJ 

•  MM 

JL 

m 

M  M  M 

M 

M 

M  M  M 

i 

p«*  M  M  M 

M 

MMM 

M 

M  M 

_ 

M 

MMM 

M 

M 

M 

M 

JL 

•  •  •  I  •  I  » 


R*  U  U  U 

X  X  X  X 


Ol 


7  tfl  -O 


3  fi  -J  — 

UOQ  O 

X  9  — 


J  o 
o  — 


86 


(UM  INUtu  j 


3 

(U 


« 

co 


o 

x 


o  •  o 
•—  cr  ^ 


u  — 

•»»« 
-4  -I  3 

«ii  _J  CO 

o 


o  P- 
JC  J 


—  r- 
o 


f*.  o 

5T  • 
•-  O 


m  ■  t  •  •  B 

w  •  »  •  t  | 

CO  /*  I  OOOO  t  9  0  0  0  •  0  3  3  0  ft  OOOO  ft  ^  fs. 

Ui  •  •  •  •  •  •  •  •  |  •  •  •  •  •  •  •  |  • 

— i  I  OOOI  OOOI  OOOI  OOOI  o 

»  •  •  I  I  » 

3  MMMn  1  *  *  * 

•  ••»""  T 

•  •  •  •  I 

“  aiaoooiooooiooaoiooooimm 

—4  •  •••»  •  •  •  |  •  •  •  •  •••!««■ 

•  OOOi  OO  O  t  OOOI  OOOI  u 

—  <n*  •  I  I  , 

"•3  •  •  I  .  . 

~t  »  •  I 

■till 

—4  IOOOO  |  •ocyN 

J  •  •  •  •  *  •  •  •  •  •••■  •  •  •  |  IT  • 

^  •  OOOI  o  —«  O  B  O  3  I 

Ui  J  |  ^  »OJ  •  o  IPO 

3  I  B  • 

J  1  #  B  **  T 

M  t  C  |  |  | 

**  — *  1X1  •  —  O  f*.  BOlPOObO  I  —  O  C*  f»  BOOOO  I  f\|  ^ 
44  B  •  •  •  I  •  •  •  |  ••••  *  *  *  I  i/>  • 

Xu.  I  O  «*0  I  »OfO-«»  O  ~  O  B  OOOI  ir» 
•—  B  o  I  o  i  ru  B  b 

*  w  i  -  i  |  «  # 

-3  •  B  »  B  g 


^  BOOOO  t  O  5  I  O  O  O  O  I  OOO 


I4J  -  I 

I  « 


O  O  r\|  I 


o  o  o  • 


*-uy  u  » 
<£  i  x  x  i 

->  B 

a  i  _j  »->  i 
o  o  o  a  i 

OCU^  I 


J  o 

o  *— 


o 

X 


o 

3 


o 

a* 


u 

Ui 


J 

►— 

CO 


X 

o 

u 


i. 

CO 


Ui 

X 

o 


^  o  o 

3  O  <7 


U  JC 

o  ^ 

ai  O 
^  J 

#*  JN 
«—  vXJ  <k> 

•  at  m 

<r  ik  • 

O)  N 

-j  t  cr 
^  J  <x> 
ai  po 
jy  CJ  33 
*r*  ru 

Q  It  • 


vj  c* 

u.  Jj  *«  II 
0X3 

X  3  > 

XI  W 

3  cO 

O  X  »-•  - 

3  r  x 
cr  x  o  uj 
3  -  x 

-•  <  X 

^  r  3 


3 

X 

ui 

a 

X 

3 


87 


88 


fcL2  tit  *2 

CUM  NULL  1 NG  F  UK  ,  , 

SU6J2  VALUt,,  2  NGN  ACCT 


NU^btH  OF  PISSING  CeStHVATIOKS 


AD-A152  800 
UNCLASSIFIED 


AN  ANALYSIS  OF  THE  EDUCATIONAL  REQUIREMENTS  OF  AIR  2/2 

FORCE  COST  ANALYSIS(U)  AIR  FORCE  INST  OF  TECH 
URIGHT-PATTERSON  AFB  OH  SCHOOL  OF  ENGINEERING  P  PERRV 
DEC  84  AFIT/GSN/LSV/84S-24  F/G  5/9  NL 


£ 


►  .  ■ ' 

l  ■  _  . 


to 


II  am,*  ilk*  ^  I  -W 


,*\  . . 

■l.^n,a-’ 


.  1-. 


(COMlNUeO) 


£13 

>  UMRUU1NG  FOR 
SUBJ3 


w-.  »rv  >rv  « v  *  _w  v  '■v*'  v  ^  ^  —  r-  -i  *v~  * 


-v 


3 

*xi 


o 

x 

a. 


« 

CO 


K 

u 


0) 

CO  J> 


CO 

3 


u* 

-I 

*-  Ui 

<o 


Ui 

CO 


OK  U. 
»—  UJ 


^uuo 

2  a.  a.  x 

-3 

3  2  -I  »- 

u  o  o  o 

X  U  F- 


■om^ 

'O 


i  ir»  r»* 

•  ■  • 

I  9  O 


o  o  o  o 


0  0  0  3 


o  o  o  o 
•  •  • 
o  o  o 


in  9  * 
m 


©  o  o  o 


©  ©  o  o 


0  9  0  0 


O  l 

u» 


J  3 
ui  a. 


a. 

w 


o 

3 


(3 

UI 


r\l 

tn 


u 

Ui 

u 

2 

9 

u 


2 

'3 


»  —  □  CO 


a.  ui 
x  ui 
mw  ui  at 


<  o 

« 

CO  Ui 
-J  Ui 
m  —  j  a: 


.  < 
►  ^ 

I  3 


O  -* 

M  Oi  O 


U.U  £ 

O  2 
Ui  -O 
^  J  (D 
X  J  - 
ui  si 
‘•X  9 
Oik  • 

9 

ji/iin 
j  in 
Ui  fx 
1/1  U  9 

in  m 
ON  • 
Ui  O 
*—  Ui 
U  X 

lkii<  II 

a  u.  3 

X  3  > 
f-  UJ  CO 
3  CO 

a  a.  *+  - 
3  x  x 
ox  uu 
Jl«  X 
XXX 
-<  <  X 


.URUER  OF  RI39INC  CeSfcRVMI0N9 


i? 

vV 

A.V 

i 


:  rj  v'L'TT'r* 


1  IP"  ■JP'I*1^ 


o 

«K 

a. 


CO 

O 

X 

o 


I* 


O 

u. 

O  9 


ac 


m 

m 

(J 

« 

U4  « 

a 

« 

x  « 

u 

« 

« 

X 

« 

lA. 

« 

9 

o 

« 

« 

«  < 
a  N- 
x  a 

X 

9 

M 

r* 

X 

• 

o 

•  -* 

4i 

3  « 

o> 

— J 

CO 

*— 

>- 

«  « 

UJ 

O 

-J 

<« 

« 

UJ 

CO 

-J 

« 

3 

3 

« 

a 

«r 

UJ 

-J 

« 

—  UJ 

r-  v) 

*— 

« 

—  J 

a  u. 

r-  ui 


HUUU 

x  a.  a.  x 

-j 

3  X  -J  »- 

o  a  o  o 

X  O  ^ 


B 

>0  90  1 

•  •  •  • 
OOOB 
t 
• 
B 


o  o  o  o 


«o 

m 


B 

B 

OOO  B  -«<MON 

•  •  •  t  •  •  • 

O  O  O  B  —  O  O 

•  in 

B 

B 

t 

B 

r*  tn  t  r»  «n  to  o 

•  ••!««  ••• 
^90  •  ON- 

•  rtIN- 

B 


B  B 

B  B 

n  9  -  9  |  N  o  »o  O  I  onitA  7 

•  ••I  •••19  •  •  • 

-OK)  B  K)  9  I  O  9  N 

f*  B  l/>  -  I  9  0^ 

B  B 

B  B 

8  B 

B  B 

-KI^)N  B  M  9  O  9  I  9  71AO 

•  ■  |  ■•••■«  ••• 

rvj  o  •  »n  m  —  b  (VN  k) 

i  -  b  r\i  9  — 

B  B 

B  B 

B  B 

t  B 

or*  B  Kl-o  -  B  OAJO- 

■  •  I  •  ■  •  I  •  •  • 

«n  o  ■  fnin^v  b  no  9 

B  f\»-  B  PO 

B  B 

B  • 


•  B  B 

B  •  • 

•4  0  O  N  |  O  O  O  O  »  o  O  O  O  B 

•  •  •  I  •  •  •  B  •  •  •  • 
9  0  0  1  OOOB  OOOB 

in  B  B  B 

•  II 

B  •  • 


B 

• 

•  O  O  B  r\J 

•  •  •  B 
»  O  O  B 

B 

B 

B 


B 

B 

90CVN  B  O 
•  •  •  B 
9  BMV  B 


B 
B 
■ 

|  M  M  —  M  —  —  I 
B 
8 

•  or*—  •  o  o  ■ 
•  •  •  •  • 
OOOB  O 

n*  s 

B 

B 


B 

B 

OOO  •  • 

•  •  •  B 

OOOB 
B 
B 
• 


>008 

>  •  •  I 
>008 
B 
B 
B 


•  OOO 

•  •  • 

OOO 


B  B 

f  B 

B  OOO©  B 

i  B  •  •  •  B 

B  OOOB 

B  • 

B  • 

I  I 


B  • 

•  • 

i  «  m  9  i  f\iro  9  a 

•  •  •  i  •  •  •  i 

Kl  AJ  K>  |  O  in  "4  • 

nj  —  •  -o  • 

B  • 

•  * 


I 

I 

10  0  9  B  — 
•  •  •  ■ 
WlpKI  | 

ai  nj  • 


KION  f  o  ©  • 

•  •  •  B  • 

lAlAO  B  Ol 


B 

B 

>  o  in  —  • 

•  •  •  B 
ON  fM  • 

O  fl 

B 


B 

B 

O  B 
•  B 
O  B 
B 


i  m  i 
»  • 

►  m  < 


B 

B 

8  nj  — 

i  ru  • 

b  in 

B  — 

B 

B 

— 

B 

B 

•  fxim 

>  i  «  • 

b  m 

B  CO 
B 

• 

— 

B 

B 

■  o  9 
I  9  * 

B  N 
B  NJ 

B 

B 


—  O  O  r*  8  ON. 

•  •  •  »  fU  • 


— j  a 
a  ♦- 
u 


3 

u> 


92 


ICCMIMifcU) 


t  L  <t  dY  R4 

^UMHULLI^G  FOR.. 

SU8J4  VALUE,  t  4  F ACRCEC 


O 


ft  • 

•  I 

J>  ft  9  900  I  0< 

•  ■  •  •  I 

•  00  9  1 

•  ft 

ft  ft 

ft  ft 


•  ft 

•  I 

OO  ft  <9000  ft 


0  9  1  9  OO  ( 

ft  • 

ft  ft 

ft  K 


•  • 

ft  • 

I  9  10  0  0  0  1 
I  •  •  •  •  » 

-  Ui  ft  O  O  O  ft 

■  CO  I  I 

•  J  •  ft 

I  ft  ft 


•  ft 

ft  ft 

>o  o  o  i  oooo  i  oo 

•  ••ft  •  •  •  §  • 

ooot  OOO •  o 

•  I 

ft  « 

ft  ft 


ft  ft 

ft  • 

o  o  «  o  o  o  o  •  r\*  — 

•  •ft  •  •  •  t  <m  » 

OO  |  OOO  ft  to 

ft  ft  — 

ft  ft 

ft  ft 


ft  ft 

ft  ft 

ry  a  -  04N  ft 

a  •  •  •  a 
ft  -»  o  • 

I  Ad  ft 

ft  ft 

ft  ft 


ft  •  ft 

moi  o  i  -»o  ftN  «  oooo  •  <\im 

•  •  •  ft  •  ■  •  |  •••■40* 

o  -•  o  «  ooot  f\i 


ft  ft  ft  ft 

Ui  ft  ft  ft  ft 

X  W  <V  ft  WOW)  -  ft  NO  O  9  ft  — NUnN  ft  o 

wU  -9  ft  •••*  •  •  •  ft  •  •  •  I 

QCU.  ft  iP  r*»  r\j  I  O  IP  — •  ft  ONO  ft 

-vU  I  ^  »  o  ft  -  ft 

M.  VJ  ft  ft  •-  ft  ft 

itD  I  ft  ft  t 


ft  ft 

ft  ft 

OOO  ft  -  OIPN  ft  OO 
•  ••ft  •••19  • 

ooo»  onto  i  p- 

ft  O  ft  t>» 

I  ••  ft 

ft  ft 


ft  I 

ft  I 

—  ft  OOOO  I  OO  I 

ft  •  •  •  ■  • 

ft  O  O  O  ft  o 


ft  ft 

I  ft 

ft  WOO  -  ft 


OlAOl  I 
IP-  ft 
ft 
ft 


-UUU  ft  o 
4X1^  I 
J  ft 

3  C  J  ►-  I 
<->  a  o  o  i 

I  U  —  I 


or 

-4  3 


o 

CL 


19 

3 


o 

X 

u 


31  OUT  OF  60  (  66,31)  OF  THE  VALID  CELLS  HAVE  EXPECTED  CELL  FREOUENCY  LESS  THAR  5,0 
'IMFUP  EXPECTED  CELL  FREGUENCY  *  0,014 

<  Ah  CHI  SUUARE  »  52 ,52499  hi  TR  44  DECREES  CF  FREEDOM.  SIGMFICARCE  »  0.1772 

CKAPEH'3  V  s  0,29990 


tLS 

COMROUJhG  FOR 


U"> 

41  • 


•  R 

9i  i  a 

■O  /I  a  3  0  0  0  ft  O: 

uu  I  •  •  •  R 

4  •  9001 

iu  •  a 

9)  |  R 

J  •  » 


•  • 

•  • 

*  O  »  Al-O?  R  O 


•  » 

•  • 

0  0  9  I  9990  R  O 


9  0  0  C  M  9 


.91  ^  O  «  » 

•  OR 

•  —  • 
•  » 


9  0  0  1 

< 

I 


•Of  -* 


I  »  •  I 

•  ft  •  • 

•  oooo  t  a  oo  o  •  n  -  • 

•  •  •  #  »  •  •  *  i  •  •  •  ft 

•  ooot  OOOR  nofti  » 

•  ■  »  -*x>  • 

•  «  •  9 

tilt 


I  t 

ft  § 

rs  i  oooa  • 

»  •  I  •  •  •  i 
I  O  I  OOOR 

R  R 

R  • 

•  R 


>  O  O  R  <U  «— 
»  •  •  R  • 

‘  O  O  R  *T 

R 

• 


9  •  •  R  R 

•  I  I  •  I 

l  \n  J*  9  9  I  -I/IN  »  I  I  f*  O  fU  «  9 

r  •  •  •  i  rv  •  •  •  i  •  •  •  »  •  •  •  l 

R  J>  O  rO  f  r-  v/1  9  |  ^  I 

r  9  —  r  ki  9  *•  r  m-  R  rst  •*  R 

•  9  R  •  • 

R  R  R  •  I 


ft  •  ft  I  R  ft 

R  R  ■  »  •  • 

i  i  Kifnr***  •  t  cr-  cr  —  rvi  »  o  r-  o  <£>  i  r«.«  »  o  • 

I  •  •  •  R  (M  •  •  •  |  •  •  •  R  •  •  •  R  •••R 

R  r»  9  •  O  O  r»  I  0901  min-R  f  N07  I 

f  rv  V  9  9  ~  t  9  R  m  *•  R  N«*  R 

•  R  R  R  R  • 

R  R  I  •  R  9 


I  R 

I  R 

►  o  i  -*orvjr»  r 

»  •  •  •  •  •  i 

•  OR  O  ru  O  R 

R  IP  • 

r  R 

I  I 


>  O  O  R  4(0 

*••19  • 

►  O  O  t  -• 

R 

R 


<or>  l  9  « 
•  •  I  4  • 
••OR  r*> 

•  9 
R 

• 


R  R 

R  t 

V  f*i  r*>  r*  -•  R  y 


•+  ~  fM  l  o  r^. 


•  R 

t  R 

*«  I  9444  R 


R  R 

t  t 

i  4  N  R  *-0  4  N  R 


r-  O  <V  R  4  Ml  4  t 

rv  ••  i  *•  m  r 

R  R 

«  • 


9  9  1/11 
rv  (V  R 

R 

R 


o  m  o  • 
m  r 

R 


*—  *—  i—  R 
^  uuu  •  •• 
x  (L  x  i 
J  R 

O  »  -I  *-  R 
UOOU  R 
X  I 


(tCNUKueu) 


COU*K 


« 

• 

• 

K  • 

• 

• 

• 

• 

*•—  ►—  1 

«  m 

UUU  1  - 

(M 

a  <r 

ai 

it 

« 

X  -* 

m 

X 

x  x  a.  i 

X  « 

«  X 

3 

i 

-3  *— 

• 

◦ 

3 

S-»*-  1 

-1  O 

«  UL. 

u 

0  0  0  9 

o  *— 

X  O  *-  ft 

w 

«  O  r* 

_k 

z  -> 

3 

«  r-.  —  23 

a* 

J  J  J 

3 

. 

<  ^  J  (O 

X 

3 

«  X 

*» 

»— 

X 

3 

3 

z 

«  X 

LLJ  CJ 

X 

3  15 

u 

<-> 

X 

3 

'3 

• 

«  _j 

98 

• 

V 

-  •  - 

•  *  - 

«l~-  *-  ’  -  - 1  - 

-  _  i  -  -v'  V 

.• 

tu  bl  K  6 

IHlH  L  Ir<G  f-UH,. 

SbbJfl  V  A  L  U  t  a  «  6  KG  I  A  K  0  Ctl-AVIOH 


— J 

r*» 

47* 

CD 

tfl 

—  HI 

— «  n 

HI 

•« 

5T 

flD 

ru  sr 

■a- 

o 

• 

mm 

• 

r»  • 

HI  • 

• 

■ 

• 

• 

o  *- 

a# 

a* 

mm 

ru 

ru 

am 

mm 

o 

X 

a 

cy 

ru 

o 

- 

- 

- 

- 

- 

- 

M 

- 

— 

- 

- 

••■**-•  h* 

- 

M 

M 

M  M 

~ 

M  M  H 

- 

- 

CO 

CO 

n 

H| 

O  >•  1 

O 

O 

O 

O 

^  -M  O  - 

o  o  o  o 

O 

0 

0 

0 

o  o 

o  o 

0  0  3 

o 

d 

CO 

Ui 

• 

•  »  • 

• 

• 

» 

•  •  • 

•  •  m 

•  •  • 

• 

•  • 

•  • 

• 

• 

-1 

n  o  i 

o 

o 

O 

^  n  ru 

o  o  o 

o  o  o 

o 

o  o 

o  o 

o 

ru 

Ui 

«•* 

ru  « 

r» 

CO 

3 

Ui 

x 

ru 

Ui 

p*»  -x  # 

ru 

— 

r-> 

Ui  ui  HI  - 

>»  47“  ST  «i 

o 

0 

0 

0 

o  o  o  o 

0 

0 

0 

o 

in  hi 

-> 

• 

•  •  % 

• 

• 

• 

ry  •  •  • 

•  •  • 

•  •  • 

•  • 

m  • 

«* 

HI 

• 

r— 

Ui 

Ui 

m  —  » 

mm 

in 

mm 

ru  — •  ru 

o  o  o 

O 

o  o 

o  o 

o 

40 

"U 

mm 

m  rm  ~m 

— •  mm 

ru 

mm 

— > 

-l 

HI 

— * 

i 

— ■ 

u 

a*- 

o 

&■  X)  o  — 

O  hi  i/i  ru 

ru  r>~  4*  =r 

— * 

o 

=T  r«. 

ru  o  cr 

<y* 

cr 

_> 

• 

■  •  § 

•u 

• 

• 

• 

ru  •  •  • 

«-*••• 

•  •  • 

• 

•  • 

•  • 

• 

■ 

u. 

— •  O  1 

mm 

in  r- 

3^13 

r»  hi 

O  ru  -• 

n 

-•  o 

o  ru 

— * 

Ui 

mm 

>o 

«• 

^  si  ru 

•0^4- 

-o 

ru 

o 

CO 

-U 

X 

— * 

aj 

<\l 

■O 

^  ?r  i 

Hi  r» 

o  -  C  J* 

•o  sr  ru  ru 

o 

0 

0 

0 

HI 

o 

^  mm 

0 

0 

0 

o 

r*»  <u 

Ui 

— > 

• 

•  •  i 

• 

• 

• 

-H  •  •  • 

•  •  • 

m  m  r> 

• 

•  • 

•  • 

• 

ru 

• 

X 

u. 

r*  -*  i 

-O 

— 

rvi 

>0 

3 

o  o  o 

in 

—  ru 

O  O 

o 

CD 

ui 

"V* 

«u 

•u 

— •  HI 

—•  <U 

r» 

H 

rnm 

•A* 

<Si 

Ui 

J 

_J 

c-» 

•h 

HI 

-U  - - I 

OJ 

-u 

ru 

HI  ru  HI  ■— 

ru  n  ru  cr 

i 

1 

1 

o 

o 

o  o 

o  o  o 

o 

O  "O 

• 

*  •  • 

• 

• 

• 

•  •  • 

•  •  • 

m  m  m 

• 

•  • 

•  • 

• 

m 

—  O  1 

•u 

r\l 

•h 

I-orv 

■O  ru  mm 

HI  ^  O 

o 

o  o 

o  o 

o 

•O 

«• 

rnm  § 

IM 

HI 

ru 

HI  mm 

X 

o 

• 

• 

• 

• 

• 

1 

• 

•— 

u 

CJ 

o 

— • 

ru 

HI 

O’ 

n 

4^ 

ml 

X  X  A.  -X.  4 
-J  • 

O  C  J  K-  | 

u  o  a  a  • 
x  _j  *—  i 


100 


(LtM  IM/tU) 


no  by  he 

luMKuUING  FuH., 

s^bje  v*LLt,.  e  hci  no  t ti-Mioh 


_j 

=T  <Xj 

K> 

** 

to* 

f*. 

iT 

o 

X 

£ 

«* 

• 

• 

• 

•x 

to 

CJ 

<x 

Q 

>— 

cm 

Ol 

o 

•* 

o 

UJ 

U) 

X 

CJ 

o 

X 

to* 

to— 

to* 

X 

X 

to* 

to* 

to* 

M  *“« 

to* 

to* 

to* 

to* 

to* 

to*  to* 

to* 

to* 

to*  to*  to* 

to* 

-J 

X 

• 

toi 

J> 

cn 

» 

u« 

to* 

-O 

X* 

• 

o  o 

o 

o 

o 

•o 

o  o 

o 

0 

0 

0 

:T  cO 

(to) 

CO 

uj 

• 

to 

• 

• 

to 

to  to 

*  •  • 

to 

t 

o 

o 

o 

o  o  o 

o  o  o 

'M 

o 

<u 

• 

UJ 

to 

GO 

1 

*— 

X 

-3 

• 

CJ 

a 

to* 

to* 

■4  to* 

to* 

to* 

to* 

to* 

to* 

to*  to* 

to* 

to* 

to*  to*  to* 

** 

uJ 

c-3 

• 

X 

Ui 

• 

JJ 

3T 

• 

o  o 

<=> 

o 

•* 

m 

u*  r» 

o 

c 

0 

0 

If)  K1 

UJ 

X 

• 

• 

• 

• 

to 

to  to 

to  to  • 

ro 

• 

UL. 

UJ 

• 

a 

o 

o 

*1 

CM  O 

o  o  o 

x 

r— 

iO 

• 

rO 

CM 

;» 

a- 

to* 

— » 

• 

«» 

(J 

— i 

• 

to* 

to*  *-* 

M 

to* 

i 

to* 

to*  to* 

to* 

to* 

to*  to*  to* 

to* 

X 

CO 

a 

CO 

Mi 

a 

X 

UJ 

— 1 

pn 

a 

GT  O 

<o 

33 

o  o 

O  O 

o  o  o  o 

O 

O 

—1 

<x 

-> 

■ 

• 

• 

• 

to 

•  • 

to  to  to 

o 

• 

Uj 

a 

X. 

a 

o  cn  cm 

o 

O  O 

o  o  o 

f*- 

CJ 

UJ 

UJ 

• 

o 

■5T 

c. 

71 

a 

•* 

3  X> 

-> 

a 

»-*  nj 

o 

•■* 

** 

•-4  *-• 

•— • 

to* 

to* 

to* 

to* 

to*  to* 

to* 

to* 

**  to*  to* 

to* 

J  o 

rO 

a 

• 

xl 

a 

Jto 

o 

X 

-J 

rvi 

a 

o  o 

o 

o 

r\j  r-» 

to* 

o  r*  r*- 

fX. 

*J 

-J 

a 

• 

• 

• 

to 

to  • 

to  •  • 

Oi 

to 

JJ 

X 

X 

a 

o 

o 

o 

-o 

r*  «-* 

o  m  o 

<X> 

X 

II 

X 

— 

xi 

• 

o 

o 

•* 

to* 

w 

CO 

• 

to* 

>* 

to* 

XI 

a 

CJ 

£ 

30 

** 

►* 

M  to* 

to* 

to* 

to* 

to* 

to* 

to*  to* 

to* 

to* 

to*  to*  to* 

to* 

O  X 

X 

_i 

• 

*J 

M 

U 

• 

-P  r* 

to* 

'J 

« 

0 

0 

o 

o 

o 

o  o 

o  o  o  o 

M  c3 

toto 

to* 

• 

• 

to 

• 

to 

to  to 

to  to  to 

to 

a 

1X1 

to- 

a 

o 

o 

o 

o 

o  o 

O  3  D 

-O 

•  'X 

to* 

a 

■o  *- 

• 

X 

X 

a 

■u 

cr- 

*x» 

o 

a 

— J 

cr 

cn 

•“to  to* 

to* 

•* 

•— •  to* 

to* 

to* 

to* 

-*  to* 

to* 

to* 

**  to*  ** 

—• 

—1 

=T 

-U 

i 

a 

• 

a 

*J 

O' 

CJ 

•— 

— *■ 

• 

o 

CJ 

*— '  CJ 

o 

CJ 

a 

o 

CM 

«n 

X 

-J 

cT» 

CM 

CJ 

x  x 

X 

X 

a 

— 

-* 

X 

«l 

CJ 

it 

• 

X 

J  C  J  h-  I 

u»  o  o  o  » 
x  u  *—  • 


CJ  ac 

U.oi  <(  II 

o  Q.  -P 

X  J  > 
Ka*  50 
3  CO 

o  x  - 

p  r  r 

l*>  X  CJ  Xi 
X 


101 


tLlb  dt  M5 

C  Uf-  I  Kill  L  U.G  f  oh  ,  , 

StfiJlb  VALUt,.  IS  LIN  Mlh  NGCtlS 


NUNbtH  Of  HISSING  CLSthVA I  IONS 


1 14 


UlM  IMjEU) 


113 


B 


I 


0 


o 

cc 


06 


<J>  — 
»  2  T 
—  —  CD 
_t  -l  J 

J  CO 


3 


O 


O 

A 

X 


-ft 

IP 

IP 

o  o 

3  r*» 

o 

O 

4P  IP 

■o 

Aft 

Aft  3 

fA 

o 

£  4 

• 

mm  • 

o  • 

— « 

• 

• 

• 

a 

3 

• 

□  *— 

za 

r*» 

—  <v 

K1 

3 

mm 

«•* 

o 

X  o 

o 

*• 

•* 

** 

*m 

M  M 

• 

>*4 

**•  m  mm  mm 

*■* 

** 

*-• 

*—»  M 

mm 

M  M  M 

*■» 

K*  M 

►u 

T 

CO 

• 

CO 

p 

•4 

o 

■a  Na 

• 

—  0  *0  r*» 

'POO*  — 

— 

O  -A  N» 

O 

O 

o  o 

mm 

Z 

? 

Z 

o  o 

o 

o 

N* 

<7* 

UJ 

• 

•  • 

• 

•  •  • 

•  •  • 

•  •  • 

• 

•  • 

•  •  • 

• 

• 

• 

• 

_J 

o 

3  O 

1 

0  3  0 

aj  Aft  M 

0  3  0 

o 

o  o 

0  3  0 

o 

o 

o 

3 

UJ 

nj 

• 

mm  mm 

3 

*«  mm 

mm  mm 

CO 

1 

.o 

• 

• 

ft 

UJ 

3 

fO 

o 

mm  —a 

• 

7  O  li<  V 

(3  <U  7  Q 

K> 

O  •* 

o 

3  I— 

mm 

f*  3  r* 

o  o 

o 

o 

«v 

3 

-1 

• 

•  • 

ft 

•  •  • 

rA  •  •  ♦ 

•  •  • 

• 

•  * 

•  •  • 

• 

• 

• 

3 

• 

►*-  UJ 

ft 

o  tr  aj 

O  -  -* 

orsnj 

OfMO 

•D  Aft  O 

o  o 

o 

cr 

—  CO 

o 

1 

3 

(Mrs 

m 

AJ 

mm 

Aft 

—  -J 

• 

-• 

• 

• 

1 

-J 

rA 

««u 

o 

®  h» 

• 

7  O  ^  B 

pu  3  o  3 

— 

O  BN 

3 

O 

—  CD 

Aft  K>  -O  3 

-m  O 

CD 

N- 

a 

Aft 

o 

• 

•  • 

ft 

•  •  • 

7  •  •  * 

•  •  • 

• 

•  • 

•  •  • 

• 

• 

• 

ip 

a 

u. 

o 

-*  o 

ft 

ONB4 

®  IP  3 

0  —  0 

o 

»•->•  AJ 

fA  *A  «• 

o 

mrn 

o 

3 

UJ 

aj 

• 

3 

3  r»  aj 

mm 

CD 

IA 

Ip 

rA 

-o 

ft 

-J 

ft 

• 

ui 

ft 

X  -i 

aj 

o 

o 

o  o 

ft 

-OWN 

ro  —  r*  mm 

Ol"  B 

O 

O 

o  o 

w* 

NAN 

^  o 

PA 

r» 

o 

o 

-u  — * 

• 

•  • 

ft 

•  •  • 

OJ  •  •  • 

•  •  • 

• 

•  • 

•  •  • 

• 

• 

* 

rA 

• 

X  u. 

o 

o  o 

ft 

O  K>  O 

Aft  -D  <0 

O  W  (M 

o 

o  o 

<3  K»  O 

o 

PA 

o 

^a 

—  UJ 

ft 

mm 

r\j  f*-  — 

3  — 

-p 

AJ 

X  UJ 

ft 

Jj  -3 

• 

mi 

ft 

** 

V 

u 

•4 

o 

o 

o  O 

ft 

3  0  0  0 

■O  O  O  AJ 

•u 

O  IP  N* 

o  o 

o  o 

— 

N  N 

o  o 

O 

o 

D 

•O 

MM 

• 

•  • 

• 

•  •  • 

•  •  • 

•  •  • 

• 

•  • 

•  •  • 

■ 

• 

• 

• 

o 

o  o 

ft 

o  o  o 

JllO  7 

o  rvj  o 

o 

o  o 

■D  Aft  O 

o 

o 

o 

4P 

** 

ft 

N* 

mm  mm 

■a  *w 

X 

• 

cu 

V 

ft 

ft 

ft 

ft 

ft 

ft 

ft 

■—  r— 

►— 

U  <-J 

vu 

cv 

r> 

3 

CD 

O 

AJ 

*C 

— J 

X  X 

X 

-** 

X 

O  S  J  I 

u  a  o  a  i 

XU  -  • 


_»  Q 

C  *- 


» 


112 


M^aUaXlAji 


(CCMINUED) 


tLli  Mi 

cun  I  Null  ing  for.. 

Slib  J 1 1  VALUt,.  13  Ml  I  Lib  ALGtURA 


COLIM 

TOTAL 


(COM  JMlfcU) 


CUMNouInC  for.. 

SIB J t l  VALUt,.  II  FRODICTN  ROT 


SM0im«3S03  DMISBlrf  JO  HJOjOs 


VALUt . ,  10  PGT  JKFO  StST 


*  w  rji 1 


"l"  THWTF  ■ 


p 


C/3 

fa 

Ck3 

OS! 


O 

X 


-J 


0> 

O) 


o 

o 

• 

fa 

o 

o 

cn 

II 

CM 

m 

II 

>* 

CJ 

zn 

CO 

2 

H 

2 

fa 

M 

O 

S3 

3: 

w 

O' 

H 

fa 

— 

< 

fa 

> 

fa 

O 

fa 

o 

fa 

fa 

CO 

fa 

• 

fa 

fa 

o> 

CN 

lO 

O 

cj 

m 

O 

vO 

o> 

o 

o 

II 

• 

m 

2 

fa 

OJ 

M 

H 

W 

II 

• 

CO 

O 

02 

CO 

fa 

< 

X 

hi 

II 

M 

cu 

o 

CJ 

2 

X 

O'  Q 

2 

> 

fa 

CO 

hi 

< 

fa 

hi 

CJ 

CO 

O 

2 

1— * 

fa 

w 

• 

o 

fa 

fa 

fa 

fa 

2 

CJ 

M 

fa 

fa 

H4 

fa 

2 

S 

fa 

2 

O 

O 

<r 

h-H 

< 

M 

fa 

2 

fa 

CO 

U 

2 

-» 

-o 

M 

—  r*» 

I*Y  O 

ru 

—  r- 

f- 

rx  9 

ao  o 

^  <* 

• 

• 

• 

• 

• 

• 

• 

ST  • 

O 

ST 

o 

ru 

— 

o 

o 

— 

—  o 

x  a 

9 

*- 

•• 

** 

M 

**  **  ** 

•J 

M  —  —  M 

**  M  M  — 

M 

H*  11  M 

MH«t1  *1 

— 

MM  — 

— 

M  M  M  — 

M 

GO 

XI 

9 

9  9  0 

9  9  9  0 

9  9  9  0 

9 

0  9  9 

ft 

0 

0 

0 

9 

0 

ft 

0 

9  0  0  0 

ru  9 

UJ 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

m 

-J 

<9  9  0 

<9  0  9 

9  9  0 

9  9  9 

9  0  9 

9  0  9 

0  0  9 

— 

IU 

<n 

o 

U4 

ST 

a 

C 

0 

0 

0  9  0  0 

9  9  0  0 

9  9  0  0 

0  9  0  0 

9 

0 

0 

0 

o  o  o  o 

n-*  in 

-1 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

—  • 

P—  Ui 

o  o  o 

o  o  o 

o  o  o 

o  o  o 

0  0  9 

9  0  0 

o  o  o 

— 

►—  <o 

— 

—  — i 

-J 

-i 

P-1 

0  —  0 

—  o  o  ?*» 

m  r»  — 

— 

9  o  n 

9  9  9  9 

9  0  9  9 

9  9  0  0 

o  — 

Z3 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

T  • 

LL. 

9  9  <M 

9  rvi  o 

«  T  -* 

9  AJ  9 

9  9  9 

0  9  9 

9  0  0 

It* 

IP 

o 

■O 

IP 

P-1 

CO 

— 

-3 

U1 

X  -I 

rx 

r MWT5T 

o  o  o  o 

-fONP* 

9 

0 

0 

0 

— 

ONN 

0  9  0  0 

®  rvj 

■AJ  _» 

•  •  • 

■  •  • 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

XI  • 

X  u_ 

f-1  r-1  — 

o  o  o 

H>  —  O 

o  o  e 

0  —  9 

0  —  0 

o  o  o 

O' 

*—  ui 

K1 

O 

o 

r+> 

*  CM 

— 

— 

ul  3 

-J 

J 

— 

— 

r*  ji  r* 

0  0  9  0 

9  9  0  0 

— 

Olfll*. 

0  9  9  0 

O 

9  0  9 

ftio  —  T 

A|  IT* 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

M 

>0  T  O 

0  9  9 

90  9 

0  9  0 

9  9  0 

9  0  0 

O  — 

m 

— 

IP 

o 

— 

X 

— 

e7.l*)  OF  IHfc  VALID  CELLS  MVE  EXPECTED  CELL  FhECUfcFCY  LESS  THAK  5,0 


(COM  IHifcU) 


X 


cr 


u 

« 

-4 

r-  cm 

<X3  — 

fO 

o 

tn  <s 

<M  CM 

mm 

r»» 

P*. 

O 

£ 

<* 

• 

r>»  • 

• 

•4 

• 

— • 

• 

• 

• 

• 

« 

o  >— 

m 

K> 

CM 

<3 

00 

o 

o 

mrn 

o 

X 

3 

M 

m 

9 

X 

a* 

« 

*-«. 

mm 

X 

• 

a 

•  « 

p-t 

»-• 

»-« 

M  M  M 

H< 

M  M  M  M 

*— 

M  »4 

-4 

mm 

-4 

— •  M 

«-• 

mm 

— 

— «  — 

mm 

mm 

mm 

mm  mm  mm 

mm 

*-• 

mm 

mm  mm 

u4 

■ 

— * 

13  « 

CO 

• 

-A 

y> 

j> 

V* 

r*  o  r» 

—  fO  o  r- 

0  0  9  0 

o 

o  o 

o 

o 

o  o 

O 

» 

9 

9  9  0 

9  O 

9  9 

CM 

■» 

>■ 

<«.  « 

UJ 

•  •  • 

•  •  • 

•  4 

• 

•  • 

• 

•  • 

• 

t 

•  •  • 

• 

•  • 

• 

n 

-4 

nj  o  o 

—•0  3 

o  o 

o 

0 

c 

O 

o  o 

o 

B 

0  0  9 

9 

O  O 

mm 

« 

iu 

in 

m 

ft 

to 

• 

-3 

« 

-3 

• 

-j 

* 

• 

i 

03 

« 

lu 

<r 

m  ir  — 

a;  w  a  ^ 

PO  O  U>  o 

CM  — 

C3- 

o 

o  o 

o 

• 

o  o  o  o 

a 

O 

O  o 

CM  O 

3 

•  •  • 

•  •  m 

•  • 

• 

•  • 

• 

•  • 

• 

• 

•  •  • 

• 

•  • 

CM 

m 

« 

UJ 

3  C  41 

o  uj  m 

on  fv 

in  cr 

— 

0 

0 

o 

• 

o  o  o 

o  o  o 

m 

CO 

CM  ^ 

—  rt 

o  — 

mm 

« 

— • 

+— 

« 

•— 

3 

—4 

• 

-a 

• 

<o 

« 

** 

•— » 

M  W  *U 

*-« 

►4  M  *4  -4 

►4 

►4 

mm  mm 

r* 

mm 

mm  mm 

•u 

mm 

mm 

M 

mm 

t 

mm 

mm  mm  mm 

mm 

mm 

mm 

M  mm 

M 

tn 

« 

ft 

-4 

m 

m 

—  — •  CO 

—  43  cO  <T 

0 

0 

0 

o 

CO  J1  CO 

ST 

IPPH  7 

T 

• 

mm 

o  i nr* 

9 

O 

O  © 

9 

«i 

o 

« 

-3 

— • 

•  •  • 

cr  •  •  • 

•  • 

• 

m  m 

m 

•  • 

• 

1 

■  •  • 

• 

m  m 

9 

■ 

X 

iA  <T  » 

c%J  O  f** 

O  O 

o 

mm  mm 

»n 

— •  fH. 

f*y 

• 

o  -•  o 

o 

O  O 

43 

X 

« 

UJ 

K1  -HI 

in  o  cm 

43  mm 

B 

o 

CO 

B 

mm 

3 

« 

43 

• 

« 

B 

UJ. 

B 

« 

« 

X 

3  CM 

o 

0 

3 

8 

«D  —  CM 

o 

o  o 

o 

— • 

r»  -• 

r- 

O  3 

o 

B 

O 

o  o  o 

O 

o 

o  o 

<M  V 

uu 

3 

— • 

•  •  • 

—  •  •  • 

•  • 

• 

•  • 

■ 

•  • 

• 

ft 

•  •  • 

• 

•  • 

«1 

• 

« 

« 

X 

X 

K>  43  CM 

o  o 

o 

r»»  m 

o 

in  <r 

CM 

• 

o  o  o 

o 

o  o 

M 

H— 

3 

CM  *0 

fMin  — 

CM 

B 

CM 

« 

« 

J*. 

(/J 

• 

ui  J 

t 

-I 

w 

B 

« 

« 

B 

4) 

-4 

o  o  o  o 

U 

9  r— 

(C|UN 

• 

o 

9  9  9 

— • 

o 

m 

wi 

43 

« 

« 

mm 

•  •  • 

— •  •  •  • 

•  • 

• 

•  • 

• 

•  • 

• 

• 

•  •  • 

• 

•  • 

CM 

• 

© 

O  r*  r\| 

CM  PH  43 

o  o  o 

in  co 

m  r»  cm 

B 

9  0  0 

9 

cr  o 

J1 

« 

X 

— u 

«—  — H 

•*  « 

m* 

PO  — 

B 

o 

—4 

X 

B 

mm 

« 

« 

. 

^  3 

mm 

»»4 

M 

»* 

J!, 

mm  — «  -4  — 4 

i 

M  M 

i 

mm 

mm 

mm 

t-m  mm 

mm 

• 

— « 

— • 

mm  rnm  mm 

i 

rm 

•4  V* 

• 

►-►-*—  I 
u  u  u»  •  -• 
4C  X  1  X  I 
-3  B 

□  f  J>-  » 

U  Q  O  (J  I 
•X  <3  *—  I 


•  t 

CM 


»n  r»  <r  X  -j 


-J  o 
o  *- 

3 


o  cr 


J  J  J 


© 

3 

vaj  3  a. 


X  3  3 

U  3  X 


102 


aCNUNUtU) 


1  w\  ir- .1  jyv  •  F  •  V";  '/>■ — 


■if1.1  "L"'  !J  1 


—  « 

3 

u>  «© 

fa 

«• 

r-  cm 

CM  5T 

•a  fa 

io 

aa 

to 

<o 

O. 

— «  • 

"O 

te 

•a 

• 

a 

• 

• 

• 

« 

O 

o 

•o 

fa. 

tea 

o 

CM 

in 

-o 

X  3 

— 

rvi 

?r 

X 

— 

« 

X 

• 

w 

•  « 

tea 

*— « 

•m  aa  m  m 

tea 

tea 

tea 

aa  tea 

tea 

tea 

aa 

tea 

tea  tea 

tea  tea 

tea 

aa 

m  aa  tea 

tea  tea 

aa 

tea 

aa 

tea 

tea 

tea 

tea 

tea 

tea  tea 

tea 

3 

1 

a 

a 

a 

a 

a 

►a 

3  « 

CO 

a 

l 

i 

a 

a 

a 

3 

CO 

XI 

a 

rO  O  O  — 

N^iN 

M 

O 

O 

9  a 

O  O 

O 

O 

•  o  o 

o  o 

a 

O 

O 

O 

O 

a 

o 

o 

O  3 

a 

f- 

>► 

«*.  « 

t4i 

a 

•  •  a 

• 

•  • 

• 

• 

•  a 

a 

• 

a 

•  a 

a  a 

a 

• 

a 

• 

a 

• 

•  a 

• 

3 

3 

a 

O  <M  CM 

CM 

JJ  O 

o  & 

tea  | 

o 

o 

o 

a  o 

O  O 

a 

o 

O 

o 

a 

o 

O  O 

a 

«* 

•* 

a 

cm  ^ 

•a 

<M 

a 

a 

a 

• 

a 

CO 

a 

a 

a 

a 

a 

a 

3 

« 

— 

• 

— 

aa  M  aa 

JL 

— 

►a  tea 

tea 

tea 

a 

tea  tea 

tea  aa 

aa 

tea 

a 

aa  aa 

a 

tea 

aa 

a 

tea 

— 

tea  -a 

a 

aa 

o 

o 

J  T 
—  —  CD 
J  J  J 
oi  J  (O 


I  I  • 

•  •  a 

a  inmnitfi  «  ^  m  o  w  •  wi 


a 

I  —  OJ  •  — •  C 3  |  9000 


a 
t 

OOOO  KVOIT3 


I 

1—0* 


3 

a 

•  •  • 

a 

• 

• 

• 

a 

CM 

•  •  « 

a 

• 

•  • 

1 

• 

• 

a 

a 

• 

• 

• 

a 

•  •  • 

o 

• 

a* 

« 

UJ 

a 

fO  CM  co 

a 

CM  3 

a 

-3  3  3 

a 

o 

r\i  o 

a 

o 

o 

o 

a 

o 

o 

o 

* 

lO  3  « 

to 

»—  kO 

a 

ro  — 

a 

cm 

"VI 

a 

ro  *n  -a 

a 

m 

a 

a 

• 

CM 

CM 

— 

« 

•a 

3 

a 

• 

a 

a 

a 

a 

a 

3 

a 

a 

a 

a 

a 

a 

• 

co 

•* 

tea 

tea 

tea  tea  tea  tea 

tea 

tea  tea 

tea 

aa 

tea 

tea 

tea  tea  tea 

tea 

►a 

tea 

tea  tea 

tea 

tea 

tea 

tea 

tea 

►a 

tea 

tea 

tea 

tea 

►a 

tea  tea  tea  tea 

tea 

a 

a 

• 

a 

a 

a 

a 

co 

« 

a 

a 

a 

i 

« 

a 

a 

3 

ro 

a 

in  »o  cm  in 

a 

—  flO 

^  cO 

• 

tete 

CO  C7*  *> 

a 

— 

O 

=T  T*te 

a 

aa 

O 

=3-  fa 

a 

tea 

rO 

fa 

l 

m  in  cm  m 

CJ 

* 

W 

a 

•  •  • 

a 

CM  • 

• 

• 

a 

fn 

•  •  • 

a 

• 

•  • 

a 

• 

• 

• 

a 

• 

• 

• 

a 

•  •  • 

« 

• 

Ute 

• 

o  r*»  ro 

a 

■a 

o 

» 

O  7  a 

a 

o 

—  o 

• 

o 

aa 

o 

a 

■o 

tea 

o 

a 

CM  r*-  ro 

x> 

X 

« 

OJ 

a 

ro 

a 

in  ro 

tete 

a 

7T  ^  CM 

a 

m 

a 

o 

a 

ro 

• 

-O 

CO 

a 

i 

■ 

a 

• 

a 

t 

a 

o 

* 

3 

a 

« 

a 

a 

a 

• 

• 

« 

a 

a 

a 

a 

a 

a 

a 

-jj 

a 

a 

a 

a 

a 

i 

a 

« 

« 

X 

3 

CM 

a 

OOOO 

a 

3  CM 

o 

CM 

a 

a* 

70W 

• 

O 

o 

o  o 

a 

o 

O 

O 

o 

a 

CM  fa 

O 

a 

-iroN 

O 

tea 

3 

3 

a 

•  •  • 

• 

• 

• 

• 

a 

•  •  • 

a 

• 

•  • 

a 

• 

• 

• 

i 

• 

• 

• 

a 

•  •  • 

CM 

• 

« 

« 

X 

U_ 

a 

o  o  o 

a 

o 

a 

ro  m  3 

a 

o 

o  o 

a 

o 

o 

o 

t 

o 

aa 

a 

cm  m  o 

sr 

Hte 

3 

• 

a 

«a 

-o 

t 

—  ^ 

a 

a 

a 

•o 

aa 

a 

aa 

aa 

« 

« 

Wi 

• 

a 

a 

a 

• 

• 

a 

3 

3 

I 

a 

a 

a 

a 

a 

a 

3 

a 

• 

a 

a 

a 

a 

a 

« 

« 

•« 

a 

a 

a 

a 

• 

a 

• 

3 

•a 

a 

a 

o  o 

o 

O 

9 

cm 

0  0  3* 

t 

o 

0 

0 

0 

• 

o 

o 

o 

o 

a 

O 

o 

O 

O 

a 

OOOO 

x»  in 

* 

<  « 

aa 

a 

•  •  • 

a 

• 

• 

• 

* 

•  •  • 

• 

• 

•  • 

a 

• 

• 

• 

a 

• 

• 

* 

• 

•  •  • 

• 

*—  O  O  VJ  •  -a 

a.  x  x  a 


3  s  — 
UOU  Q 
X  3  — 


J  — 
3  3 
O  — 


116 


(tCM  iKUtU) 


cn  • 

a)  i/>  •  ^  r 


»  • 

•  • 

^  f  Ai  <*»  — «  -ST  $  O’  * 
I  •  •  •  t 
•  •  « 

B  — •  B 


f  x>  —  o*  -o  I  .O  O  : 

•  •••!•*  •  • 

»  l>*  M  1 

l  3  -  •  o  ru  . 

•  l 

l  l 


l  « 

>u  i  I 

*.  ~J  CVJ  |  T  1/1  HI  O  •  1/1  1/1  HI  ?r 

Ui-J  B 

Xo.  B  r<i  CO  CVJ  •  CO  —  o 

*—  jj  t  oi  c  pn  f*i 

X  W  I  B 

w  3  *  f 


I  • 

•  t 

^  I  B  dl  —  'M 


B  ««  Oi  -«  I  J1  'll  • 

I  ••m  •  rvi 


•  II 

l  I  I 

13  3  0  0  1  0090  t 


O  O  O  f 

I 


0  0  9  1 

B 

■ 

» 


•  09  |  ki  -* 

»  •  •  l  • 

•  091  nj 

• 


t  • 

i  i 

•  9  »  9999  «  — •  < 


OJ  (T  IT  B 

r*jy  b 

t 

» 


O  O  O  I 

I 

• 


o^ot 

ru  I 

i 

I 


»  —  m  r-  » 

•  •  •  »  (V  • 

—  a  O  I  1/1 

—  I 


•  I 

»  • 

09  I  —  O  «0  f*»  I 


3  • 

jy  ST  CM  t 
I 
I 


Jl  *•  O  I 

ru  • 

I 

• 


ID-IIM- 
»  •  •  »  « 

'  »  -'ll  I  f 

I  •  J 

I 


•  I  •  • 

«  I  B  I 

>  K»  •  I  HIOHI**  I  (VOfM^  B  1/1  «  9  IT  B  CD  I 


o  rvj  r*>  b  o<anjt 

-•  AJ  V  O  • 

»  ••  • 

t  • 


—  B  l/IOHI  ( 

»  in  -h  « 


9  O  B  0009  B 


B  B 

B  B 

I  0  0  9  0  B 


—  I 

HUUU  B  — 
4C  3.  X  a.  B 

J  t 

a  5  -I  ►-  I 

UOOU  I 
X  U<-  I 


(tCM  IMiEU) 


£L  1 7  BY  h  1 7 

CUMHOLUNG  FUN  ,  , 

Slib  J 1 7  VALUt ,  ,  1 7  BASK 


V  *  LU  t 


COLLHK 


ELIO  BY  Mb 

CllK  1  HULL  1NG  Fuh,, 

SLBJIO  VALUt , ,  \t  Ccbrl 


X  7 
—  J) 


Oft  Ul 
-X  — • 


CO  I 

n  tn  •  o  o  o 

Lift  I  •  • 


o  o  o  t 


•  I 

•  ft 

9  I  o  o  o  o  ■  o  < 

•  ft  •  •  •  • 


o  o  o  I 


O  O  i  O  9 

t  •  •  |  • 


•  ft 

ft  ft 

I  a  I  -«  o  I 

»  •  •  •  •  ft 

■  Ui  I  O—OI 

•  co  i  in  i 

»  j  i  i 


•  • 

•  • 

»  O  9  O  ft  3  0  00  t  ®  O 

•  •  •  »  •  •  •  t  m  • 

o  O  O  I  OOO  I  'O 
ft  19 

ft  ft 

•  • 


•  •  ft  • 

>  •  ft  ft 

u  w  i  *»o  vn  i  —  o  cr  f»  coooo  (mm 

-i  •  •  •  •  •  •  •  •  i  «•«•</>• 

U.  ft  0—01  0  —  01  OOOft  O 

•  n  •  tn  i  i  w 

Vi  •  •  •  ft 

->  ft  ft  I  ft 

ft  ft  ft  ft 

*1  t  ft  ft  ft 

ft.  JOJ  ft  9000  t  —ONfv  ft  — or^r*  ft  x»  — 

Oft  _ft  ft  •  •  •  t  •  •  •  •  •  ••*—• 

X  ■  OOOft  O  9  O  ft  0901  — 

uj  *  ft  «n  to  »  — 


o  o 
9  a 
u.  a. 
X  aft 
aft  i£ 


-1  X 

Oft  o 

U>  aft 

O 

a  sy 
—  r** 

J  o  r\i 


w  3  -O 
un  aft  r-> 


A-  U  U  U  ft  o 

x  x  x  a.  «  — 

-»  ft 

O  £  _»  —  ft 

u  a  o  o  • 
x  o  —  • 


UI  X 

a.  aft  <  ll 
□  a.  j 
x  J  o 
—  u*  CO 
3  n 

a  >.  >— •  - 

3  x  x 


I  V  9 

9  9  9 

3  0  0  I  -  O  O  9  O  O  O  O  9 

•  ••9  •  •  •  «  •  •  •  | 

O  O  O  I  -•  O  O  I  0009 
9  0  9  • 

9  •  I 

9  9  9 


MO  O  O  I  OOOO  I  O  O  O  O  9 

•  ••I  •  •  •  9  •  •  •  I 

0  0  0  9  0009  0009 

9  9  9 

I  9  I 

9  9  9 


a  ft  9  9 

till 

Ui  ^  90000  90109099  4)  IT  O  -  9  O  O  < 

J  9  •  •  •  «  •  •  •  |  •  •  •  9  • 

*-*  Ui  9  0009  «  IP  ^  I  tflir»99  o  « 

»—  </J  9  I  —  r\|  |  r—  9 

•  ■  9  9 

-19  9  ■  9 


9  9  9 

9  I  I 

'  O  •  OOOO  I  OOOO  9  o  o  . 

'  •  t  •••»  •  •  •  I  • 

-09  0009  0009  o  < 


99999999 
9  •  9  I  |  9  •  9 

9  I  OlOilA^r  I  <Al  o  O  9  -*1*^011^.  I  OOOO  I  OOOO  I  —  oi  r-  ■  ru 

9  •••9  •••IOJ  •  •  •  |  •  •  •  |  •••9  •  •  •  I  •  *  •  I  fO  • 

I  O  -Q  *+  t  9  <0  x  I  I  Orooa  0  0  0  9  0009  -*1^0  I  - 

•  O  I  9  f\j  rv  |  I  s  t  -  9<M 

»  -•  »  9  I  t  9  9  • 

9  1  1  9  9  9  9  9 

9  9  9  9  I  I  I  I 

•  9191199 

^^Ol  9  OOOO  I  I  N  9  f\i  fO  9  Oim  O)  9  I  OOOO  9  «  C  C?  N  •  9  9  N  N  9  OJ  «0 

9  •••#  *  •  .  |  ro  •  •  •  |  •  •  •  t  •••a  •  •  •  I  •  •  •  *  k/1  • 

•  0  0  0  9  VT  O*  I  —  in  9  W  m  |  0  0  0  9  0^0  9  9f^<M  I  'J> 

•  9  7  9  r*>  ru  I  <+*  9  9  J>  •  9  9  •*> 

99999999 
99199999 


•  9  9 

9  9  9 

to  9  f\inj  r*  9  I  i/iNO  o  • 

'  •  ■ 

tO»  «w«*9  99  9  1 

R  9  mflfM  I 


9  19 

9  9  9 

4  -•  9  -•  O  f*  9  O  9  r«-  9 


o  jirti  i 

in  i 


in 


t-  —  •—  9 
f-UUU  9  — 
£.  X  i.  A.  I 
J  • 

3  *  -I  t-  I 
(->0009 
X  Jh-  9 


122 


-cum  iNueo) 


U2U  dY  K20 

CUM  Hill  l  I  NG  FOR,, 

SueJ20  VALUt,,  20  HANAGRL  STATS 


►—  LJ  U 

«£  a.  jl  a. 

a  c  j*- 
o  a  o  o 
iu  ^ 


»  » 

9  I 

0  O  O  O  O  I  -03 


»  I 

•  » 

^  t  3  3  0  0  |  O  « 


I  I 

•  I 

3  10  9  3  0  1^: 


0  3  3  1 

I 


»  o  o  o  • 

I  I 

•  • 

9  9 


O  O  O  I 

I 

• 


»  0 

•  0 

t  i 


0  I 

I  0 

-  •  O  OO  O  I 


Mnn  i 

in  • 

0 

0 


O  O  O  I 

0 


•  0 

0  I 

i  o  o  ■  o  o  o  o  «  cr  cv 

»  •  •  t  •  •  •  |  • 

O  O  0  OOOf  « 

I  • 

•  • 

0  0 


ill 

•  0  0 

0001 P  •  J>  5T  FO  IPjp^qr^  •  3  O 


•  0  I 

•  •  I 

9  O  •  O  O  O  O  •  9  0  9  0  I  N  JT 


®  O  «P  » 

pi  pi  i 

I 

• 


» 

ru  4P  —  • 

0 

0 


0  0  3  1 

I 

0 

0 


3  0  0  1 

I 

I 

0 


0  0  0 
111 

-0-TV  O  I  N  r\J  PH  IP  ■  B«M|p  |  «i  | 


N  | 

J>  PI  -*  0 


PI  9  <©  0 
7  7-1 


•  O  •  Ip  9  Pi  I 

0  N  0 


t  ■©  r-  i  —  ® 

►  •  •  0  9  • 

•  ®  0  — 

i  •  or 


ir^9  -  l  f^0(M 


I  PI  PI  Jl  PI  0  -*  O  •"  T**  0  «*0**F—  f  PJ  >  f 


9  «  -  I 

Pi  in  t 


orvo  I 

Jl  0 

0 


Jl  PI  O  1 
PJ  0 


I 


UCM  lMjtU) 


U2U  BY  B2Q 

LUMhULLlUC  FUR., 

SUB J2u  V  ALlit ,  ,  20  RAMUHL  STATS 


1 

** 

*“4 

mm 

M  M 

m  mm 

mm 

mm 

*-4 

mm 

T 

CO 

1 

;/> 

T* 

1 

mm 

r- 

O  P*- 

O 

o  o 

o  • 

O 

O  o 

O 

O 

O 

o 

o 

UJ 

• 

• 

•  m 

•  • 

•  1 

•  • 

• 

• 

• 

• 

-1 

1 

•o 

3  O 

0 

0 

O  1 

0 

0 

o 

o 

Uu 

• 

mm 

ift 

co 

• 

• 

• 

• 

OJ 

z* 

1 

O 

O 

o  c» 

a 

u>  C3 

o  » 

o 

o  o 

o 

o  o 

o 

o 

-J 

• 

• 

•  • 

•  • 

*  • 

•  • 

• 

• 

• 

• 

r— 

Ua 

• 

O 

o  o 

o  o 

o  ft 

<=>  o 

o 

o 

o 

o 

>— 

<0 

1 

_> 

*. 

-i 

• 

1 

• 

-J 

ro 

1 

mm 

r- 

p»-  p*. 

o 

o  o 

o  • 

o 

O  3 

o 

o  o 

o 

o 

-> 

« 

• 

■  ■ 

•  • 

•  t 

•  * 

• 

• 

• 

• 

u- 

1 

0^3 

o  o 

3  • 

o  o 

o 

o 

o 

UJ 

« 

mm 

-o 

1 

1 

• 

■u 

1 

£ 

— i * 

rv. 

ft 

f\J  K> 

mm 

w  -or*-  i 

•• 

i*»  xj  r**- 

o 

o 

o 

o 

J4 

-> 

• 

• 

•  • 

•  • 

•  * 

• 

• 

• 

• 

ac  u~ 

ft 

l*> 

rP  — 

•*>  — ■ 

O  | 

vO  — 

o 

o 

o 

o 

*— 

<44 

t 

r*\> 

*n 

• 

mm 

Y/J 

• 

• 

sAJ 

• 

ft 

-1 

• 

I 

1 

• 

U> 

1 

'VI 

m  cr 

ry  r>.  -o 

CT  | 

/>  'O  a 

mm 

o 

— • 

r*» 

ft 

• 

•  • 

•  • 

m  • 

• 

• 

• 

• 

A— 

ft 

cr  -* 

■a  cr 

««  | 

.*>  O 

o 

rw 

G» 

»■* 

• 

♦n 

■o 

ft 

OJ  mm 

o 

X 

• 

ft 

u> 

• 

ft 

• 

» 

ft 

ft 

« 

t— 

•— 

ft 

UJ 

UJ 

UJ 

• 

c 

oj 

cn 

3. 

a. 

3. 

t 

— • 

** 

** 

a  s  _i  ►- 

u>  o  u>  u 
X  •— 


•X  » 
tnik  • 


«aJ  lp 

x*  u  r- 

in  ru 

£3  »•  • 

4-*  O 


o  a.  j 
-<  i  > 

*-  1*4  lO 

^  y> 
o  x  - 
3  r  x 

U  iaJ 

^  —  X 


.■.  :J 


-uRbfcH  OF  KISSING  CtStRVATlONS 


SLbJil  V  A  LUt ,  ,  21  REGRESS 


•  I  I  I  I  I 

(OS  S  I  ■  I  I 

v>  inioooo  »^oor»i3oooioooo»ooooi-*N. 

Ul  S  •  •  •  ft  ■  •  •  I  •  •  •  S  •  •  •  9  •  •  •  S  • 

-J  •  0  3  0  1  jy  o  o  •  o  o  o  »  o  o  o  »  ooos  o 

UJS  f  OJ  o  t  l  I  I 

(OS  •  —  •  S  I  s 

J  «  •  f  I  s  ft 

•  t  «  »  a  s 

s  S  I  s  s  s 

Ui  I  O  <3  O  O  I  OOOO  «  O  O  O  O  I  OOOO  lOOOOftfl? 

_J  S  •  •  •  ft  •  •  •  I  •  •  •  I  •  •  •  I  •  •  •  |  • 

•-  uj  s  ooos  ooos  ooos  ooos  ooos  m 

*—  <o  t  t  s  s  •  I 

•  s  s  s  s 

-J  e  I  t  c  a  ft 

s  I  s  I  a  • 

•  •  i  i  s  a 

-J  ^S-*OCON  I  OOOO  IOOOO  IOOOOIOOOOI 

-3  s  •••a  •  •  •  a  •••a  •  •  •  •  *  •  •  »  rv  • 

vJL.  ft  JlrOOS  0008  OOOS  OOOS  OOOS  f- 

ui  a  ru  t  s  a  a  •  — 

cn  i  a  a  t  a  a 

3  »  a  a  a  a  a 

a  a  a  a  a  a 

ui  a  a  a  a  a  a 

i  jot  i  a  a  imo  0D9  aro(7*f*»  —  aoooo  i  k\-o 

ui  j  a  •••a  •••a  •••a  •••a  •••am* 

x  a.  a  o  »  io**o  i  o  m  -*  a  fMinnt  a  0009  <0 

»—  uj  a  x*  a  <\»  1  ip  asr  a  am 

x  w  1  a  a  a  a  a 

ui  j  9  a  a  t  a  a 


a  a  a  a  a 

a  a  a  9  9 

a  —  o  ■—  r*-  a  Aio-O'j  1  f\j  o«n  »  »  t  ^  ^  o  a  nj  : 


*/>  -•  o  a 
01  a 


I 

in  a 


►  u  l»  u  a  (7* 

4.  X.  U.  X  t 

J  I 

3  e  *-  a 

UUU  U  I 

X  U  -  I 


'IMRuR  EXPtClEC  CEIL  FREGUENCY  s  0,007 

"An  Chi  SUIARE  a  64,73760  W 1 1 H  44  DEGHEES  CF  EREEDOF,  SIGNIFICANCE  s  0.0224 
CmAEEN'S  V  s  0.31414 


tt.22  UY  N22 

LUMRUlLlNG  FUR.. 

SUdJ2<;  V  ALUt .  .  22  NEGRESS  II 


F  1  V 


▼ 


I  L  n  M  |I>  v !J I II  pp  ■>!  Pill  up  m  PVI >|«  ■>!  >■« 


1 


-I 

a 

o  ® 

—4 

n 

AJ  AJ 

AJ  3 

~4 

r««- 

-o 

AJ 

■3- 

o 

«* 

• 

5  • 

3 

• 

A 

• 

• 

• 

• 

3- 

• 

o 

aj 

X3 

AJ 

o 

5T 

— « 

o 

CL 

a 

AJ 

AJ 

AJ 

o 

*— 

*• 

** 

*■* 

• 

^ 

-»  M 

FA 

1 

"■* 

H*  (-» 

B 

F>* 

*— • 

• 

-* 

• 

*— « 

■— •  **« 

• 

# 

• 

■ 

B 

b 

B 

Cl- 

3 

• 

A  —  r** 

ix'  in 

r—  lH 

• 

A 

» 

A 

3 

3 

ft 

o 

w 

o 

O 

B 

o 

o 

o 

o 

o  o 

o 

B 

— i 

s 

•  •  • 

• 

•  • 

1 

•  •  • 

1 

• 

• 

• 

1 

• 

• 

• 

B 

• 

• 

• 

•  • 

• 

B 

• 

UJ 

• 

a  r-  o 

r\i 

in  fa 

• 

—  nj 

» 

<T 

l\J 

B 

<3 

o 

o 

B 

o 

o 

o 

0 

0 

o 

B 

</> 

• 

a 

— • 

A 

• 

AJ 

B 

AJ 

ft 

B 

B 

-J 

• 

• 

B 

B 

B 

B 

-A 

• 

1 

B 

B 

ft 

B 

B 

• 

B 

ft 

B 

B 

1 

B 

t 

1 

1 

ft 

_J 

A 

• 

a  a>  r*» 

AJ  © 

A  A 

1 

AJ  A  A  A 

B 

in 

■Q 

<r 

in 

• 

o 

O 

o 

o 

1 

© 

o 

o 

o 

AJ 

A  -O 

B 

'O 

o 

-3 

• 

•  •  • 

—  • 

•  • 

1 

«■» 

•  •  • 

B 

• 

• 

• 

B 

• 

• 

• 

ft 

• 

• 

• 

•  • 

• 

B 

A 

• 

«A_ 

t 

•orvio 

O 

A  JO 

• 

CT-  A  33 

B 

in 

A 

A 

B 

o 

o 

o 

B 

•3 

o 

o 

A  J> 

B 

in 

ua 

1 

a 

FA 

A 

• 

AJ  A 

1 

—• 

1 

B 

A 

B 

Aj 

n 

1 

B 

B 

ft 

B 

B 

-3 

• 

B 

B 

1 

B 

B 

B 

1 

1 

B 

ft 

B 

-u 

C 

f 

1 

ft 

B 

• 

-J  aj 

• 

A  o  n- 

in  in 

3  3 

1 

AJ  A  in  A 

• 

«■— 

3 

■O 

>o 

ft 

— 

o 

o 

r- 

B 

O 

o 

A 

o  cr 

B 

* • 

3 

-u 

— » 

B 

•  •  • 

• 

•  • 

• 

»4 

•  •  • 

B 

• 

• 

» 

1 

• 

• 

• 

B 

• 

• 

• 

•  • 

• 

B 

in 

• 

cr 

ix 

1 

W  »\iO 

n» 

cr  o 

S 

cr  a  ® 

B 

3 

r*. 

B 

o 

AJ 

o 

• 

o 

AJ 

O 

o  m  aj 

B 

m 

»■- 

W*J 

• 

A 

A 

AJ  -• 

B 

AJ  AJ 

B 

A 

AJ 

B 

m 

B 

o 

n 

B 

A 

M 

w 

• 

B 

• 

B 

B 

— • 

B 

Jj 

1 

1 

« 

B 

B 

B 

-1 

• 

« 

1 

B 

ft 

B 

• 

B 

B 

B 

B 

B 

• 

o  o  o  o 

*>  O 

6 

6 

B 

3 

—  -O  F- 

B 

A 

■a  aj  o 

B 

o 

A 

r-» 

t 

© 

o  o 

o 

>»■ 

A 

F^» 

B 

A 

cr 

*• 

• 

•  •  • 

• 

•  • 

• 

M 

•  •  • 

B 

•• 

• 

• 

• 

» 

m 

• 

• 

S 

• 

• 

• 

•  • 

• 

B 

3 

• 

— 

• 

o  o  o 

© 

« in 

B 

3  aj  cr 

B 

o 

O 

cr 

B 

O 

AJ 

© 

ft 

o 

o 

o 

"O  AJ  O 

B 

cr 

* 

AJ 

B 

A  A 

• 

cr 

A 

t 

m 

B 

B 

AJ 

£ 

• 

B 

B 

B 

B 

B 

CJ 

B 

B 

B 

B 

B 

B 

1 

« 

B 

B 

B 

B 

B 

B 

• 

<_» 

— > 

t 

— • 

AJ 

A 

3 

m 

■o 

as 

-21 

«J 

*1 

£ 

i. 

a. 

• 

X. 

-J 

• 

-J 

F— 

o 

I 

-1 

1 

-J 

o 

UI 

a 

o 

o 

• 

O 

x 

sJ 

B 

CJ 

AJ 

Aj 

-J 

x. 

3 

3 

*J 

J 

iX 

J 

o 

CJ 

u 

IX. 

128 


(CCnl  I  Mjfc  1) ) 


U2<:  in  H22 

CuN I  NULL  ING  NUN ,  , 

SLbJJi  VAGUL.,  li  NfGNISt)  1) 


TTT 


x 


3 

-O  rvj 

*  9 

AJ  3“ 

j*  n 

AJ 

ar 

*  ■« 

• 

m 

• 

• 

• 

3  *- 

=T 

'AJ 

mm 

XI 

— 

Ct  13 

*"* 

• 

**** 

>— • 

mm  t—  >—* 

•— » 

M  ^ 

*-• 

• 

ft 

i 

OJ 

• 

«■*  r«- 

x  r- 

—  O 

r- 

a 

0 

p 

0 

9  O 

o  <© 

O  <3 

©  9 

« 

3 

1 

* 

■  • 

m 

• 

• 

»  •  • 

• 

•  * 

• 

•  • 

i 

*“  U- 

ft 

n-  o 

IP 

r- 

o 

9  0  0 

O 

<3  9 

9 

O  © 

» 

•—  40 

• 

AJ 

« 

~  3 

ft 

• 

3 

1 

i 

• 

ft 

1 

• 

3 

XI 

I 

nj  XI 

o  cr 

AJ  O 

■o 

a- 

o 

0 

0 

0 

o  o 

3  9 

O 

3 

3  O 

ft 

■ 

• 

•  • 

• 

• 

• 

•  *  • 

• 

•  • 

• 

•  • 

ft 

X 

1 

=3  JA 

0  3  0 

3 

3  3 

3 

3  3 

ft 

ui 

• 

XV 

OA 

• 

in 

• 

• 

3 

1 

• 

I 

• 

3 

1 

ft 

X  3 

rvj 

• 

f*. 

o  r- 

O 

© 

r- 

fM  O  »  47 

X*  o 

ar  «— 

o 

© 

O  O 

• 

3  3 

ft 

• 

•  • 

• 

• 

• 

•  •  • 

• 

•  • 

• 

•  • 

• 

(X  U_ 

1 

-O 

rvj  o 

JA 

rv» 

o 

O  XA  mm 

o 

in  A* 

O 

o  o 

V 

»—  ^ 

• 

AJ 

3 

3 

• 

X  UJ 

1 

• 

3  3 

• 

i 

3 

ft 

• 

X 

1 

ft 

3 

1 

-V*  XI 

r^-  ?r 

O  O 

o 

o 

3 

n 

0 

0 

•AJ  © 

C7 

AJ 

3 

rs.  y 

• 

1 

• 

•  • 

• 

• 

• 

■  •  ■ 

• 

•  » 

• 

•  • 

• 

— 

f 

Xft 

3- 

o 

3 

9 

9  <3  O 

3 

y  — * 

<3 

y  — • 

• 

■x 

ft 

K> 

ar 

3 

V 

X 

ft 

ft 

<3 

ft 

ft 

ft 

• 

ft 

• 

• 

ft 

»— 

1 

f— 

3  3 

3 

ft 

or 

o 

-AJ 

in 

: 

X  X 

X 

1 

— • 

~m 

—A 

» 

3 

c  3 

*— 

1 

3 

3  O 

o 

1 

X  3 

— 

• 

9  3 

3  • 

**  O 


-2 

O 


3 

3 


3 

•— « 

33 


3 

9 


•DO  —I 


JA 

r\j 


i/A  • 

in 


GU  O 

3  3 

O 

□  r*- 

i—  fO 

JOfO 


3  ju 
~  .3  *n 
X  J  AJ 


r>j  «  u. 


X  3 
4.  < 


3  a 
o  *- 


j 

o 


o 

3 


129 


J»  3 

JT  ' 

a  ii 


3  LT 

II 

o  a.  3 
x  3  :» 

►—  GU  CO 

23  CO 

a  x  -•  - 

3  z  x 
T  3L  '->  aJ 


b 


■ 


.UKbtK  OF  KISSING  CcStNVAl  ICFS 


t  L  30  0Y  h30 

CuMHULIInG  fOK., 

SG0J3O  V  A  L  L  t ,  ,  30  OKI  550 


14  3 


l  (.CM  IMjtU) 


£12*  BY  h2V 

CUMHULL1M  FUF,, 

StdJ2V  V  *LUt  a  .  ?S  lip  I  3«b 


O 

««. 

X 


3 

3 


-U  J  U  I 

^  X  I 

J  » 

3  fi  3  • 

(JOO  U  I 
-X  3  —  • 


-I  3 

O  4- 

u 


J 

3 


J 

3 

X 


3 

_ 

-4  r*. 

.4 

r». 

4  fU 

3  O 

3 

CO 

A 

X. 

• 

• 

• 

• 

3 

• 

3 

_ 

3 

© 

o 

O 

44 

o 

3 

X  U 

o 

Ui 

9 

-4 

r— 

X 

O 

3 

44 

44 

44 

—4 

M  *4 

*4 

-4  »4 

44 

44 

44 

44 

44  44  44  44 

44  44 

44 

3 

44 

3 

3 

1 

• 

• 

X 

Ui 

1 

1 

B 

3 

3 

• 

o 

©  o 

• 

o  o 

o 

3 

o 

o  oo  » 

©  o 

3 

© 

r- 

UP 

UL 

3 

1 

• 

•  • 

• 

4 

4 

• 

•  •  •  • 

• 

• 

• 

• 

U. 

• 

© 

o  o 

B 

o 

o  o 

o  o  o  » 

o 

O 

o 

Oi 

UI 

CO 

* 

• 

• 

J» 

u. 

44 

• 

• 

• 

X. 

3 

3 

1 

►4 

44 

_  ^ 

• 

44 

44  44 

_ 

4* 

44 

• 

44  44  44  44 

44  44 

44 

4* 

44 

X 

to 

■ 

1 

B 

3 

3 

1 

B 

f 

3 

U* 

3 

P*> 

1 

3 

o 

©  3 

• 

3  O 

3 

O 

o 

0  3  0  1 

3  O 

3 

3 

•AJ 

-4 

3 

X 

3 

• 

• 

•  • 

• 

• 

• 

• 

•  ■  •  • 

• 

m 

4 

ru 

• 

Ul 

o 

1 

© 

O  3 

B 

O 

3 

o 

O  o  3  B 

3 

O 

o 

m 

3 

3 

UJ 

B 

B 

B 

•• 

3 

CO 

• 

1 

B 

3 

f* 

3 

1 

• 

1 

44 

ru  o 

— • 

44 

•-4 

•-4 

*•4  44 

•■4 

44  44 

44 

44 

44 

44 

M44H4 

44  44 

44 

44 

44 

3 

o 

fO 

m 

• 

B 

B 

X 

• 

3 

• 

B 

1 

:> 

o 

X 

3 

ru 

» 

O 

ft 

o  o 

o 

o 

o 

m 

0 

0 

0 

o  o 

o 

3 

r\i  in 

3 

» 

• 

•  • 

• 

• 

• 

t* 

9  •  •  B 

• 

• 

• 

3 

• 

UJ 

X  U. 

1 

o 

•4  O 

a 

o 

O 

o 

O  o  O  » 

o 

o 

O 

ru 

X 

II 

X 

v_ 

3 

• 

o» 

B 

3 

p“ 

n 

X 

vi 

• 

B 

>- 

44 

3  3 

• 

B 

u. 

3 

c 

CO 

i— « 

►"4 

44 

•-4 

►4  »-4 

44 

44  44 

44 

44 

44 

44 

44  44  44  44 

44  44 

44 

44 

44 

O 

x 

3 

• 

B 

3  <M 

3 

X 

• 

« 

44 

3  m 

44 

3 

• 

o 

o 

o  o 

• 

-•  3 

ru 

r«4 

44 

O  r-»  r*»  t 

-4  © 

/U 

3  r» 

'J 

•“ 

44 

• 

•  • 

ft 

• 

• 

• 

•  •  9  0 

• 

• 

• 

• 

-o 

Ui 

o 

• 

o  o  o 

I 

3 

44 

o 

0  —  01 

o 

44 

o 

tr* 

■ 

X 

P4 

44 

• 

ft 

O 

©  • 

o 

m 

3 

u 

• 

X 

X 

I 

• 

*4 

•4  • 

44 

3 

3 

3 

u 

• 

a 

3 

••  =X 

CO 

*■4  p4 

44 

—4 

•4 

44 

—4 

*4  “4 

-4 

44  44 

44 

•4 

44 

44 

4NM4 

44  44 

44 

“4 

44 

w 

3 

3 

3 

<3  (X 

J.  vj  <t  II 

a  a.  _j 
*  3  :» 
^  iU  w 
3  CO 

3  X  —  - 
3  E  X 
t?  i  'J  liJ 

■O  — •  X 
^  £  4 
-4  4  X 

i  X  3 


« 

CO 


3 

X 

3 

a 

u 

3 


142 


U.29  01  H29 

CUN ) HULLING  FUfc.. 

SlibJ29  ViLUt,.  29  IH  SMS 


'  *"  *"C  1  "V  '.  V  '.TT'-'^  .  *"7C  T*3  ’.  V"P- 


I,  my  i"-i  i  iv  ii  i  ii  p  j.  '■  ll  ■■!.■  p;  *P  »l'  w  »V  V  "  *■  V  V  *  1  PJ1 


o 

Q. 


3 

3  r- 

3-  -3 

-O  ru 

-vi 

3 

3 

r-. 

3 

CM 

— 

r» 

O 

3 

JC 

• 

V  • 

CM 

• 

• 

• 

• 

• 

• 

3  *- 

■\l 

3 

M 

-M 

■O 

O 

cr 

CJ 

-O 

— 

- 

- 

“«  m 

M  >-* 

- 

MM  M  M 

M 

“» 

*-•  M  M 

M 

- 

- 

M  M 

M 

- 

- 

M  “« 

7 

- 

-- 

M 

7 

M 

M 

- 

- 

7 

Ui 

=7 

o  o 

o  o 

—  —  Or' 

CM 

—  03 

o 

o 

o  c* 

o 

O 

o  o 

— 

—  o 

r- 

o 

o 

o  o 

• 

•  • 

•  •  » 

•  •  • 

• 

•  • 

• 

•  • 

•  • 

• 

• 

• 

• 

• 

Ui 

o 

o  o 

—  in  q 

r>  o  — 

o 

o  o 

o  o  o 

—  in 

o 

o 

o 

o 

CM 

F-  <o 

•m 

in 

—  r\j 

M 

3 

3 

CM 

3 

rO 

-•  o  m  r- 

rvj  <xj  lP  ru 

n 

3  f"  3 

— • 

o 

in  p* 

M 

Ol/lp* 

'M  CM  — 

3 

o 

o 

o 

o 

** 

3 

• 

•  • 

M  m  •  • 

•  •  • 

• 

•  • 

• 

•  • 

•  » 

• 

• 

• 

• 

CM 

• 

u. 

in 

»  o 

ru  3  « 

f-  CM  m 

o 

3  3 

n 

3  3 

ru  3 

— 

o 

o 

*n 

Ui 

ru 

m  in 

—  ru 

in 

CM 

ru 

CO 

3 

3 

cm 

cm  o  rvi  3 

lOff  -o® 

o 

r*  —  ® 

C 

o 

O  O 

— 

O 

'O  r- 

ki  <o 

— 

o 

o 

o 

o 

CM 

in 

• 

•  • 

3  •  •  • 

— 

«  •  • 

• 

•  • 

• 

•  * 

•  • 

• 

• 

• 

• 

>o 

• 

cr 

o 

H>  «• • 

r-*  r\j  o 

in  -o  -o 

o 

o  o 

m 

o 

m  3 

CM 

o 

o 

o 

CM 

in 

rn  — 

CM 

•n 

c/i 

ui  3 

JJ, 

3 

1  ' 

3 

M 

o 

p*»  p» 

-o  rn  **• 

M 

noin 

M 

o 

r-  r~ 

<M  O 

3  3 

r*>  pn  Hi 

M 

o 

o 

r*» 

m 

• 

•  • 

»n  •  •  • 

— 

•  •  • 

• 

•  • 

• 

•  • 

■  m 

• 

• 

• 

• 

fm. 

in 

-•  o 

O  txi  7 

3  3* 

o 

•u  o 

O 

<-n  •< 

n  i/ic\» 

o 

— * 

o 

3 

M 

fM 

w  VIM 

r*i  — 

in 

in 

o 

n 

-C 

u 

t 

» 

• 

B 

i 

B 

B 

»— 

*— 

o 

3 

3 

«• 

CM 

3 

n 

w 

3 

— i 

^XO.O.  I 
-i  9 

OS-li-  • 
u  a  g  a  i 
3£  '3  • 


3 

ru 


-I  o 

O  ►“ 


141 


U.U.1  lMjt  J) 


tL27  By  K27 

CUMHULII MG  FUN,. 

SUHJ27  VAlUt , ,  27  SFFIMN  IK  CA 


139 


I 


o 

a. 


I 


<o 

«o 

U 


9  rg 
f\J  —  cU 


IM  JIO 
O 


o  *» 

X  9 


<x>  rg 
rg 


o  ao 


'  IO 
»  • 
9 


■X  • 
—  O 


• 

9 

— 

ft 

— 

- 

—  — 

ft 

tmm  mm 

mm  mm 

— - 

ft 

mm 

- 

w  mm 

tm* 

m  mm 

mm 

1 

mm 

- 

- 

- 

•-* 

ft 

- 

mm 

mm 

mm 

- 

3 

(A 

• 

ft 

ft 

ft 

ft 

3 

A 

• 

O 

3  0  0 

1 

O  O 

O  O 

O  O 

0  0 

ft 

mrn 

0 

O  f— 

O  O 

O  O 

ft 

O 

O 

3 

3 

• 

3 

3 

3 

3 

mm 

p»» 

«*. 

Uj 

« 

• 

•  • 

ft 

• 

•  • 

m 

•  • 

ft 

• 

•  • 

• 

•  • 

ft 

• 

• 

• 

ft 

• 

0 

m 

• 

> 

3 

• 

OOO 

• 

O 

O  O 

O 

0  0 

f 

0 

O  O 

O 

O  O 

t 

O 

3 

O 

1 

O 

3 

O 

O 

9 

1 

ft 

ft 

rg  0 

• 

• 

cn 

• 

ft 

ft 

— • 

ft 

• 

9 

• 

• 

ft 

ft 

ft 

• 

ft 

ft 

« 

ft 

• 

ft 

1 

• 

ft 

UJ 

sr 

I 

(M 

•• 

•**  «T 

ft 

O  O 

0  0 

ft 

0 

0 

O  O 

O  O 

O  O 

ft 

O 

O 

O  O 

ft 

O 

O  O 

O 

9 

3 

1 

• 

•  • 

ft 

• 

•  • 

• 

•  • 

ft 

• 

•  • 

• 

•  • 

ft 

• 

• 

• 

ft 

• 

• 

• 

• 

»-  9 

• 

m  — 

ft 

cr 

9  rg 

O 

0  0 

f 

0 

O  O 

O 

O  O 

■ 

O 

3 

O 

ft 

O 

O 

3 

3 

♦—  co 

• 

«»• 

ft 

9 

ft 

ft 

ft 

—  3 

• 

ft 

ft 

ft 

ft 

3 

• 

ft 

ft 

1 

ft 

ft 

1 

• 

ft 

ft 

• 

ft 

ft 

ft 

1 

3 

• 

mm 

0  sr 

1 

-•  O 

a*  r*» 

« 

0 

0 

O  O 

O  »** 

1 

O 

O 

O 

O 

1 

O 

3 

3 

O 

r»* 

9 

9 

• 

• 

•  • 

ft 

mm  • 

•  • 

• 

•  * 

• 

• 

•  * 

• 

•  • 

ft 

• 

• 

• 

ft 

• 

m 

• 

mm 

• 

X 

1 

—  mm 

ft 

3* 

-0  3 

0  in  0 

ft 

0 

O  O 

N  JIO 

ft 

3 

3 

O 

ft 

O 

O 

3 

«■* 

19 

1 

— • 

ft 

mm 

r-- 

mm 

t 

ft 

1 

mm 

A 

ft 

ft 

1 

ft 

ft 

9 

i 

1 

ft 

ft 

ft 

• 

ft 

ft 

ft 

ft 

UJ 

• 

1 

1 

• 

ft 

X  9 

rg 

ft 

«*• 

•• 

0  in 

ft 

cr  m 

—  m 

rg  0 

=3  a 

ft 

0 

0 

O  O 

K>  ST 

—  O 

ft 

• 

O 

f** 

r-> 

ft 

mrn 

O 

r» 

r- 

O' 

UJ  9 

t 

• 

•  • 

ft 

m  • 

•  • 

• 

•  • 

ft 

• 

•  • 

• 

•  • 

ft 

* 

• 

• 

ft 

• 

• 

• 

in 

• 

X  U. 

• 

«• 

9 

ft 

0  0 

0 

r*i  — * 

• 

0 

O  O 

m  rg 

1 

O 

mm 

0 

ft 

in 

mm 

G 

O 

•—  9 

1 

•O 

mm 

1 

«■ 

9  rg 

rg 

• 

rg 

ft 

O 

ft 

rg 

M.  I/J 

ft 

ft 

■ 

ft 

mrn 

ft 

9  9 

• 

ft 

t 

ft 

t 

9 

ft 

ft 

ft 

t 

ft 

ft 

ft 

« 

1 

ft 

9 

ft 

•*> 

r»*  O 

ft 

'M  sr 

0  to 

r-  0  o*  » 

ft 

0 

r* 

O  T 

9  9 

ft 

O 

0 

3 

3 

ft 

■0 

3 

O 

ST 

r* 

> 

• 

•  • 

• 

(•t  • 

m  m 

• 

•  • 

ft 

• 

*  m 

mm  m 

•  m 

t 

• 

• 

• 

• 

• 

• 

9 

• 

ft 

•O 

-T  rg 

• 

9 

O  «« 

0 

0  » 

ft 

0 

9  rg 

mm 

in  9 

ft 

3 

O 

O 

■ 

m 

ST 

rg 

X 

ft 

ft 

m  a’  rg 

r*» 

mrn 

ft 

<u 

r*» 

mm 

1 

• 

r* 

3 

X 

ft 

ft 

ft 

ft 

■ 

9 

ft 

ft 

1 

1 

1 

ft 

ft 

ft 

ft 

ft 

ft 

ft 

• 

►—  *• 

*• 

ft 

• 

9  9 

o 

ft 

mm 

rg 

PO 

9 

m 

9 

<*■ 

X 

9 

• 

*1 

X  X 

9 

ft 

X 

<t 

X 

3 

ft 

9 

»— 

3 

3 

■C  9 

*• 

• 

9 

3 

u_ 

9  0  9 

O 

ft 

O 

►— 

X  9 

1 

9 

138 


-  i  -  k  - 1  -  i  - 


(cci.iuuto) 


IO.M  INbEO) 


UF  F1S31NG  CeStHVMlONS 


COUfK  33 

TOUt  23.  i 


E  L  3  1  nr  H 3 1 

CUMHOLLlNG  FUN,, 

SUB J 1 1  VALUE,,  31  ON  I  SSI 


— 1 

<o  rvi 

r*» 

rvi 

9 

r* 

f\l  O 

£  <t 

• 

• 

• 

m 

• 

cr  • 

a  *- 

51 

o 

o 

o 

—  o 

x  o 

o 

*• 

**• 

t 

M  ~ 

~M 

~  ~ 

1 

<n 

t 

1 

■n 

» 

0  3  0  0 

0  0  3  0 

o 

o  o  o 

o 

O  O  O  ft 

o 

o 

o  o 

5T  O 

jj 

1 

•  •  • 

•  •  • 

•  •  • 

• 

•  •  ft 

• 

•  • 

• 

-i 

1 

oo  o 

o  o  o 

o  o  o 

o 

O  O  I 

o 

0 

0 

aj 

UJ 

ft 

co 

t 

o 

• 

• 

ft 

'UJ 

ft 

c  P. 

o  o  o  o 

C-  UJ  o 

u> 

o 

o  O  f 

O  o 

o  o 

o  o 

-1 

■ 

•  •  • 

•  •  • 

•  •  • 

• 

*  *  ft 

• 

•  • 

•Mi  • 

uu 

ft 

<0  0  0 

o  o  o 

o  a  o 

o 

O  O  ft 

o 

o  o 

r» 

r 

mm  —m 

«  — > 

• 

-i 

• 

ft 

”  *"* 

ft 

-* 

PO 

ft 

o  o  o  o 

oooo 

O 

0 

0 

0 

o 

0 

0 

0 

o 

o 

o  o 

m  <o 

-3 

• 

•  •  • 

•  •  • 

•  •  • 

• 

•  •  • 

• 

•  • 

F*1  • 

X 

ft 

0  0  3 

o  oo 

O  O  O 

o 

o  O  ft 

o 

o  o 

cr 

Ui 

ft 

ft 

f\* 

CO 

ft 

ft 

• 

ft 

I 

1 

-u 

ft 

1 

X  -J 

fM 

i 

po  o  O  -U 

•*o  r»- 

o 

0 

0 

0 

o 

!-*?**• 

wm 

o 

r-  r- 

O  K» 

UJ 

ft 

•  •  • 

•  •  • 

•  •  • 

• 

•  •  ft 

m 

•  • 

«  • 

X  U. 

• 

O  1/3  CM 

o^-o 

O  o  O 

o 

••Oft 

o 

*•  o 

EM 

—  '.*J 

i 

o 

»n 

ft 

o 

cr 

X  (/J 

ft 

• 

Ui  J 

ft 

ft 

-J 

ft 

■  i  i  i  i  ■ 

ft 

< 

• 

ft 

UI 

•• 

• 

oooo 

o  o  r» 

•u 

o 

o  r*»  ft 

O 

o 

o  o 

■n  rvi 

• 

•  •  * 

•  •  • 

•  •  • 

• 

•  •  ft 

• 

•  • 

• 

fO  Al  -j 

o  o  o 

o  m  o 

O  PO  O  ft 

o 

o  o 

• 

KT 

o 

m 

• 

X 

• 

ft 

UI 

• 

ft 

• 

ft 

ft 

1 

ft 

ft 

»>• 

ft 

o  u» 

UI 

• 

cr 

•4 

CM 

pn 

m 

X  -1 

-= 

X  X 

X 

ft 

— * 

X  «■ 

Z3 

■ 

J  f 

3 

*  -»  ►- 

ft 

-1  o 

(-1 

o  o 

o 

• 

O  N- 

XU*- 

ft 

UI 

-3 

r*\ 

JO 

jy 

co 

-3 

-1 

C3 

X 

-3 

X 

X 

UJ 

X 

o 

UI 

o 

L46 


S5  OUT  OF  60  (  S 1 , 7 X )  OF  THt  VALID  CELLS  hAVE  EXPECTED  CELL  FHECUENCY  LESS  THAF 
N1MFUN  EXPECTtO  CELL  FREQUENCY  *  0,028 

KAN  CHI  SULARE  b  4S.7C608  KITH  04  DECREES  CF  FREEDOM,  SIGNIFICANCE  b  0.2S6S 
ChAhEH'S  V  c  0.295S2 


tLJi  BY  H32 

CUMNiJlLlHG  HOB.. 


-  -w  .  ' 


9 

»9 

m 


r*>  « 

j 

ru  9 

mm 

r*. 

9  O 

3 

z 

9  9 

• 

• 

9  • 

3 

9 

M 

O  p- 

PP 

o 

•p  O 

Ui 

3 

X  9 

© 

X 

04 

« 

mm 

X 

X 

• 

mm 

•  HI 

PU 

PP 

PP 

mm  pm 

mm 

mm 

mm 

M  M 

mm 

3 

Z 

ii 

t 

3 

3 

3  « 

CO 

• 

Ui 

M 

-i 

CO 

in 

1 

o 

o 

o  o 

3 

O 

o  o 

ro  -p 

UI 

CO 

9  « 

3 

• 

• 

•  • 

• 

•  • 

■ 

3* 

-1 

I 

O  O  © 

O 

o  o 

fM 

3 

« 

3 

B 

Ui 

p 

CO 

B 

pm 

X 

« 

3 

• 

Ui 

3 

mm 

pp 

pp 

■H 

M  M 

m4 

mm 

M 

M  pp 

mm 

Ui 

3 

HI 

» 

X 

X 

• 

* 

3 

« 

uj 

9 

• 

3 

o 

o  o 

(3 

o 

o  o 

r-“  9 

UI 

X 

-1 

• 

• 

•  • 

• 

•  • 

* 

X 

« 

3 

t 

o 

o  o 

o 

<-*  <«# 

9 

Ui 

—  co 

• 

-p 

« 

•—  -i 

• 

9 

3 

« 

_i 

pp 

B 

M 

mm 

M  mm 

mm 

_ 

•M 

M  M 

X 

CO 

• 

CO 

3 

« 

• 

3 

3 

3 

HO 

• 

O 

O 

o  o 

O 

o 

o  o 

HO  9 

3 

X 

« 

3 

• 

• 

■  • 

• 

•  • 

fO  • 

Ui 

o 

l*. 

• 

O  O  O 

o  o  o 

<m 

UI 

3 

* 

Ui 

» 

Oi 

a 

CO 

• 

3 

pp 

Hi 

3 

• 

M 

rvi  cm 

M 

pp 

mm 

M 

M  pp 

mm 

mm 

UP 

mi  mm 

mm 

3 

o 

HO 

r» 

m 

• 

9 

• 

Ui 

B 

o 

« 

X  3 

rvj 

1 

up 

o 

HO  fu 

© 

o 

o  o 

9  9 

Ui  3 

• 

• 

•  • 

• 

•  • 

9  • 

Ui 

HI 

X  X 

• 

o  ru  o 

o 

o  o 

O 

X 

II 

X 

^  3 

t 

in 

•o 

p“ 

pm 

N 

m 

*  CO 

• 

J m 

mm 

ui  3 

• 

X 

3 

9 

CO 

hi 

PP 

PP 

M 

im 

mm 

M 

PU  PP 

pm 

o 

Z 

z 

3 

B 

3 

9 

o 

HI 

9 

• 

mm 

3 

HO 

mm 

(3 

•u 

• 

•« 

o 

o  o. 

mm 

o 

9  N 

ru  9 

M  3 

mm 

<t 

mm 

• 

• 

•  • 

• 

•  p 

m  • 

pp 

3  IU 

9 

HO 

P“* 

1 

o 

-•  o 

o 

mm  o 

9 

•  X  ro 

-> 

Hi 

X 

o 

■ 

in 

o 

»o 

mm 

X 

• 

X 

X 

• 

— • 

9 

9 

JJ 

Hi 

UI 

V 

3  N 

9 

CO 

•— * 

— •  «* 

o 

mm 

mm  mm 

mm 

mm 

M 

mm  mm 

mm 

3 

ru 

32 

HI 

• 

1 

B 

3 

mm 

3 

—  •— 

1 

J%  3 

ru 

Hi 

UI  u> 

UI 

• 

o 

m 

Z  3 

9 

fM 

3 

z 

a.  a. 

X 

I 

up 

pp 

X  9 

a 

M 

• 

Z 

HI 

3 

B 

3  »— 

3 

© 

ip 

a 

£  3 

« 

3  a 

3 

CO 

« 

3 

o  a 

o 

B 

O  p- 

3  X 

CO 

'VJ 

X  UI 

pm 

1 

3 

X 

3 

9 

« 

3 

F**  HI 

a 

X 

3 

X 

-5 

X 

3 

;> 

S  HI 

►— 

3  CO 

X 

3 

3 

co 

o 

co  h» 

a 

X 

■H 

• 

AJ 

3 

3  X 

X 

X 

« 

HO 

3 

3 

X 

3 

3 

3 

3 

X 

X 

9 

•p 

X 

3 

HI 

Ui 

3 

3 

Z 

9 

9 

X 

«p 

9 

X 

3 

« 

X 

3 

Z 

—1 

-«•  N 

r\i  o 

* 

• 

<r  • 

CJ  •— 

O 

o 

X  d 

o 

*■" 

•* 

**  *-4  MM 

T 

to 

ja  tn 

0  3  9  0 

mm  (*• 

U4 

•  •  • 

• 

-4 

0  0  3 

O 

O* 

CO 

-3 

uj  9 

o  o  o  o 

tn  m 

-4 

•  •  • 

• 

>-  Ui 

o  o  o 

f-  o 

-*  — > 

-1 

-l  m 

0  3  0  0 

5T  O 

d 

•  •  • 

or  • 

o_ 

O  O  O 

mm 

Ui 

<n 

oi 

i  J  nj 

o  o  o  o 

m  co 

d  —» 

•  •  • 

■a  • 

x  ic 

o  o  o 

in 

*—  -Li 

=T 

4*  V J 

ol  d 

-4 

MMMW 

Cl  « 

—•  3  r«* 

r*o 

•  •  • 

CM  • 

o  m  o 

o 

X 

«« 

Cl 

• 

^  *"■ 

♦—  d  d  C. 

in 

X  d 

x  cc  a.  i. 

X  « 

J 

J  *- 

3  <  JH> 

-J  o 

ci  a  a  a 

<3  r- 

X  Cl  »- 

Cl 

ci  o 

ua  O 

a.  su 

M.  jj 
Ui  IX 


a 

a 

M  o  00 

JO  fV 


u.  u  -X~-- 

o  X 

UJ 

~  3  l/> 
MO? 

ai^u  a 


SU  (M 
J?  » 


ui  rv 

•  U  N 


Cl  S 

tc  aj  <  ii 
d  u.  j 
M  3  > 

-3  CO 
O  X  -t* 

3  r  x 

lT3l  UU4 


*-•  <  X 
1  X  U 


tl34  UY  H3« 

CUM  HULLING  H)H., 

SUBJ34  V4Ll)t , ,  34  CM  £40 


o 

«  - 

(T  5T 

bo  r*. 

—  o 

—  P* 

an 

_ 

-  o 

£  <t 

• 

• 

• 

• 

—  • 

• 

• 

=r  • 

o »- 

OD 

O 

in 

r*- 

o 

ru 

—  o 

X  Q 

3 

** 

T 

»•* 

•>«  M  M 

mm 

*"*  —  —  — 

— 

M  ~  M  — 

7 

M  M  M  M 

M 

— 

M  —  — 

M 

on 

n  in 

a 

0  3  0 

0  3  0  3 

0  3  0  3 

3  0  0  0 

—  -•  O 

3  O  O  O  • 

3 

3  3  3 

—  Pk* 

UJ 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

•  mm 

•  •  •  • 

•  •  • 

• 

_i 

o  o  o 

0  0  3 

0  0  3 

o  o  o 

cp  o  o 

0  3  0  1 

3  3  0 

O 

Ui 

o 

(O 

o 

• 

lu  =r 

ru  o  ky  ;ar 

ip  jar  m  zr 

<?ooo 

ru  a  m  «x 

o  o  o  o 

OO  O  O  I 

O 

0 

0 

0 

-* 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

•  •  • 

•  •  •  • 

»  •  ■ 

••  • 

P»  u* 

o  « 

cr  =3-  -o 

o  o  o 

m  ^  — 

o  o  o 

o  o  o  • 

o  o  o 

tr 

»—  on 

ru  — 

•a 

ru  — 

-J 

-i  m 

-o 

o  o*  m 

O  m  m  i/l 

r\f  rvj  o  ? 

4 

0 

3 

a 

m  m  m  — 

—  O  <X>  »**.  I 

J\  o 

o 

•  •  * 

m  •  •  • 

•  •  • 

•  •  • 

•  •  • 

•  •  •  • 

•  •  • 

m  • 

-k. 

3  37 

p*  m  in 

ru  m  — 

O^iM 

r-  in  ru 

0  —  01 

■o  m  — 

ku 

3  — 

m  >o  ru 

rsi 

in 

rv 

o  • 

•O 

sn 

—  V 

o 

OJ 

X  -J  OJ 

«• 

o  o  r- 

r-  m  m  ni 

in  o  <b  in 

-mor-. 

m  in  «  in 

o  o  o  o  a 

wor* 

—  ru 

JJ  -» 

•  •  • 

•  •  • 

•  •  • 

•  mm 

•  •  »  t 

•  •  • 

m  • 

X  u. 

OIVO 

m  ru  o 

m  <p  ^ 

ru  ru  o 

o  o  o  t 

k»  ru  o 

o 

•—  -u 

— 

r*.  rv 

m 

— 

7f 

rt  vt 

Jj  3 

3 

** t-« 

11111111 

uww 

•< 

o  — 

3  O  P* 

r\i  r\*  O  0» 

—  in  o  p* 

ru  ru  o  7 

o  o  o  o  » 

3  0  0  0 

O  ru 

— 

•  •  • 

— •  •  •  • 

•  •  • 

•  •  • 

•  •  m 

•  •  •  a 

•  mm 

IV  • 

— 

o  in  o 

^  O  IP 

ru  o  — 

iv  m  o 

VO«* 

o  o  o  a 

o  o  o 

=T 

— 

— 

•«  r*- 

(V  — 

•• 

— 

— • 

z 

w  ^ 

• 

* 

i 

• 

• 

a 

a 

^  ^ 

o  o  o 

f\i 

m 

3 

m 

-o 

it 

Z  -1 

21A»IL  I 
3  I 

3  «  JK  I 
(-30001 
XUH.  I 


(ccNmuau) 


3 

3 

3 

3  «■•  3  1 

3 

—  M 

•-«  *-»  < 

— 

3  X 

t 

3  CJ 

CD 

1 

13 

CO 

1 

o 

n 

0 

0 

o  o 

o  o 

mm  f*** 

3  O 

3 

« 

•  •  • 

• 

•  • 

• 

_i 

1 

3  0  0 

o 

o  o 

o 

3 

3 

• 

3  • 

CO 

1 

3  X 

1 

JL 

_  _ 

3  3 

PM 

o  o 

3  3 

• 

X  3 

t 

-<  3 

3 

■3- 

« 

O 

0 

0 

0 

—  GS 

—  r- 

or  cr 

3  X 

3 

B 

•  •  • 

• 

•  • 

— •  • 

X 

3  3 

• 

o  o  o 

a 

f-  ® 

3 

*-  IO 

• 

o 

^  X 

3  — * 

• 

3 

^  3 

3 

B 

X 

3 

3 

3 

3  3  pm 

3 

mm  mm 

—  mm 

mm 

CO 

• 

CO  3 

• 

3  3 

-4 

ra 

1 

3 

o  so  f* 

O  O 

O  O 

x»  o 

3  X 

-3 

• 

•  •  • 

• 

•  • 

X*  • 

3  O 

a. 

1 

0  —  0 

O 

o  o 

o 

3  3 

3 

• 

in 

PO 

9 

CO 

» 

3  n* 

3 

» 

— *  3  3 

i 

rnm  mm 

•-«  PM 

mm 

•m  o  rv* 

JOIA 

O 

B 

M.  • 

3 

JU 

B 

»  O 

X  -* 

ru 

B 

3 

O  ON 

o  o 

o  o 

—  ru 

3  — » 

• 

•  •  • 

• 

•  • 

m  • 

3 

z  u. 

1 

OfMO 

o 

o  o 

•c 

X  U  X 

3  3 

» 

X! 

—  r— 

II 

M.  U) 

• 

3  3 

U4  3 

• 

3  3  X 

CO 

3 

3 

3 

—  —  1— 

mm  mm 

mm  mm 

mm 

9  X 

*- 

3 

• 

3  O’ 

O 

B 

^30 

3 

u 

3 

• 

0 

0 

0 

9  0  0  0 

O  rv 

MCIT 

3 

3 

• 

•  •  • 

m 

•  m 

«W  • 

—  3  <-> 

M. 

3 

B 

o  o  o 

O 

O  O 

•  x^ 

-» 

3 

1 

3 

•“3  • 

X 

X 

• 

cr  3 

3 

u 

1 

3  ro  r\i 

CO 

3  3  3 

3 

3 

»m 

—  —  — 

•— 

mm  mm 

mm  mm 

mrn 

3  3 

X} 

31 

B 

• 

B 

3 

JP 

3 

MB  M 

3 

B 

n  3 

n 

*-uu 

o 

B 

— • 

f\* 

X  -* 

ST 

Oi 

3 

X  X  X 

X 

1 

3 

«M 

X  < 

9  U 

• 

3 

B 

3  *“ 

3 

o 

^m 

9*3 

m— 

1 

3  O 

3 

CO 

c_>  o  o 

O 

• 

a  *- 

3  X 

to 

X  3 

3 

B 

u 

3  3  X 

IB 

3 

3X3 

X 

x  3 

3  3  CO 

X 

3 

to 

9 

ax-* 

• 

5T 

3  X 

X 

X 

•n 

-3 

'J 

3X3 

3 

3 

3 

X 

3 

or  3 

X 

9 

-U 

O 

X 

X  X 

«. 

X 

3  4 

X 

3 

>.  r 

3 

•£ 

154 


Allen,  Wayne  M.  "Towards  a  Cost  Analysis  Career  (Job)  Series."  Report  to 
DC A  Personnel  No.  DCA-P-43.  Director  of  Cost  Analysis  Office  of  the 
Comptroller  of  the  Army,  May  1980. 

Cook,  2Lt  Robert  and  2Lt  John  E.  Greene.  "An  Analytical  Study  of  the 

Graduates  of  the  Graduate  Logistics  Program  of  1965,  1966,  and  1967." 
MS  Thesis  SLSR  13-68,  AF1T/SL,  Wr ight-Patterson  AFB  OH,  1968. 

Crowder,  Capt  William  N.  and  Capt  James  A.  Davidson.  "An  Analysis  of  the 
Usefulness  of  the  Graduate  Logistics  Program  as  Perceived  by  Alumni 
and  their  Supervisors."  MS  Thesis  LSSR  1-78A,  AFIT/LS, 

Wr ight-Patterson  AFB  OH,  1978. 

Cushman,  Capt  James  H.  and  Capt  James  N.  Townsend.  "An  Analysis  of  the 

Educational  Requirements  of  a  Base  Level  Chief  of  Aircraft  Maintenance 
in  the  Disciplines  of  Production  Management,  Communicative  Skills,  and 
Statistics."  MS  Thesis  SLSR  35-71A,  AFIT/SL,  Wr ight-Patterson  AFB  OH, 
1971. 

Department  of  the  Air  Force.  Officer  Air  Force  Specialty.  AFR  36-1 
Washington:  HQ  USAF,  1  January  1984. 

"Member  Certification,"  Dollars  and  Sense,  National  Estimating  Society, 
(June  1984). 

Emory,  C.  William.  Business  Research  Methods.  Homewood  IL:  Richard  D. 
Irwin,  Inc. ,  1980. 

Goetsch,  Clarence  H.  "How  Can  the  Cost  Analyst  Be  Improved  as  a  More 
Valuable  Resource?"  Armed  Forces  Comptroller,  10-13  (May  1980). 

Goven,  Richard  C.  "Institute  of  Cost  Analysis  (ICA)."  The  Air  Force 
Comptroller,  10-12  (January  1983). 

Hart,  Lt  Col  Allan  C.  "A  Study  of  the  Graduates  of  the  School  of  Systems 
and  Logistics  Graduate  Logistics  Program."  MS  Thesis  SLSR  24-65, 
AFIT/SL,  Wright-Patterson  AFB  OH,  1965. 

Kankey,  Roland  D.  "Professional  Designation  Cost  and  Price  Analysis," 

Armed  Forces  Comptroller,  27,  39  (Winter  1982). 

Margolis,  Milton  A.  "Improving  Cost  Estimating  in  the  Department  of 
Defense,"  Concepts,  the  Journal  of  Defense  Systems  Acquisition 
Management,  4  (2):  7-17  (Spring  L981). 

Nie,  Norman  H.  and  others.  S  tat  is  t ical  Package  for  the  Social  Sciences. 
McGraw-Hill  Book  Company,  1975. 


Novak,  Theodore  J.  Jr.  and  Roland  D.  Kankey  "Is  There  a  Formal  Education 
Program  for  Cost  Analysis?"  Armed  Forces  Comptroller,  40-42  (Summer 
1981). 


Pazeretsky,  Capt  Bill,  Resource  Advisor,  Palace  Dollar  Team.  Telephone 
Interview.  AFMPC,  Randolph  AFB  TX,  8  February  1984. 

Ross,  Capt  James  L.  and  Earl  T.  Steiner.  "An  Analysis  of  Educational 
Requirements  for  Civilian  Logistics  Managers  Within  the  Air  Force 
Logistics  Command."  MS  Thesis  SLSR  20-72A,  AFIT/SL,  Wright-Patterson 
AFB  OH,  1972. 

Speck,  lLt  Ernest  E.  Jr.  "Perceived  Utility  of  the  AF1T  Graduate  Systems 
Management  Program."  MS  Thesis  LSSR  54-81,  AFIT/LS,  Wright-Patterson 
AFB  OH,  1981. 

Temple,  Glenn  0.  Initiatives  for  Improvements  in  Cost  Analysis. 

Unpublished  report  No.  47954A.  Professional  Military  Comptroller 
School,  Maxwell  AFB  AL,  1981. 


156 


VITA 


First  Lieutenant  Phillip  Perry  was  born  8  July  1959  in  Augusta, 
Georgia.  He  graduated  from  Hephzibah  High  School  in  Georgia  in  1977  and 
attended  the  United  states  Air  Force  Academy  where  he  earned  a  Bachelor  of 
Science  degree  in  May  1981.  Upon  graduation,  he  received  a  regular  com¬ 
mission  in  the  USAF  and  was  placed  on  active  duty.  He  served  as  a  finan¬ 
cial  manager  for  the  Deputy  for  Reconaissance  and  Electronic  Warfare 
System  Program  Office  until  entering  the  School  of  Systems  and  Logistics, 
Air  Force  Institute  of  Technology,  in  June  1983. 

Permanent  address:  1542  Shoreline  Drive 

Augusta,  Georgia  30906 


157 


UNCLASSIFIED  _ 


SfCURITY  CLASSIFICATION  OF  THIS  PAGE 


REPORT  DOCUMENTATION  PAGE 


1b.  REPORT  SECURITY  CLASSIFICATION 

Unclassified 

1b.  RESTRICTIVE  MARKINGS 

2«.  SECURITY  CLASSIFICATION  AUTHORITY 

3.  OISTRIBUTION/AVAILABILITY  OF  REPORT 

2b.  OECLASSI F ICATI  ON/DOWN  GRADING  SCHEDULE 

Approved  for  public  release;  distribution 

unlimited . 

4  PERFORMING  ORGANIZATION  REPORT  NUMBERIS) 

5.  MONITORING  ORGANIZATION  REPORT  NUMBERIS) 

AFIT/GSM/LSY/84S-24 

6a.  NAME  OF  PERFORMING  ORGANIZATION 

School  of  Systems  and 

Logistics 

6b.  OFFICE  SYMBOL 
(If  applicable) 

AFIT/LS 

7a.  NAME  OF  MONITORING  ORGANIZATION 

6c.  AODRESS  (City.  State  and  ZIP  Code) 

7b.  ADDRESS  (City.  State  and  ZIP  Code) 

Air  Force  Institute  of  Technology 

Wright  Patterson  AFB  OH  45433 

8*.  NAME  OF  FUNOING/SPONSORING 
ORGANIZATION 


8b.  OFFICE  SYMBOL  9.  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 
(If  applicable I 


Sc.  ADDRESS  (City.  State  and  ZIP  Code) 

|  10.  SOURCE  OF  FUNOING  NOS.  | 

PROGRAM 
ELEMENT  NO. 

PROJECT 

NO. 

TASK 

NO. 

WORK  UNIT 
NO. 

11.  TITLE  ( Include  Security  Classification/ 

See  Box  19 

12.  PERSONAL  AUTHOR(S) 

Phillip  Perry,  B.S.,  1st  Lt,  USAF 


13*.  TYPE  OF  REPORT  13b.  TIME  COVERED 

MS  Thesis  fhom _ 


18.  supplementary  notation 


14.  DATE  OF  REPORT  (Yr.,  Mo..  Day t  15.  PAGE  COUNT 

1984  December  164 


17 

COSATI  CODES  I 

FlELO 

GROUP 

SUB.  GR.  f 

18.  SUBJECT  TERMS  (Continue  on  reverse  if  necessary  and  identify  by  block  number/ 


19.  ABSTRACT  (Continue  on  reverse  if  necessary  and  identify  by  block  number / 

Title:  AN  ANALYSIS  OF  THE  EDUCATIONAL  REQUIREMENTS  OF  AIR  FORCE  COST  ANALYSTS 

Thesis  Chairman:  Richard  Murphy,  GM-13 


Tir-: 

Air  ?cicc  . 

V/ii'J-t-*  4...- - :  -  • 


vi  Vr ' 


20.  OISTRIBUTION/AVAILABILITY  OF  ABSTRACT 
UNCLASSIFIED/UNLIMITED  £  SAME  AS  RPT.  □  DTIC  USERS  □ 


21.  ABSTRACT  SECURITY  CLASSIFICATION 

UNCLASSIFIED 


22a.  NAME  OF  RESPONSIBLE  INOIVIOUAL 

22b.  TELEPHONE  NUMBER 

22c  OFFICE  SYMBOL 

(Include  Area  Code ) 

Richard  Murphy 

513-255-6289  1 

AFIT/LSY 

OD  FORM  1473,  83  APR 


EDITION  OF  1  JAN  73  IS  OBSOLETE. 


UNCLASSIFIED _ 

SECURITY  CLASSIFICATION  OF  THIS  PAGE 


UNCLASSIFIED _ 

MCUHITV  CLASSIFICATION  OF  THIS  FAOS 


The  United  States  Air  Force  has  emphasized  a  need  to  improve  the  quality 
of  Its  cost  estimates.  It  seems  that  cost  analysts  who  are  adequately 
educated  can  provide  the  high  quality  estimates  needed  by  the  Air  Force. 
This  research  effort  developed  an  educational  requirements  list  for  Air 
Force  cost  analysts.  This  list  Is  needed  to  evaluate  the  educational 
qualifications  of  not  only  potential  cost  analysts,  but  current  analysts 
as  well.  The  list  is  based  on  an  evaluation  of  a  number  of  cost  analysis 
related  subjects  by  experienced  Air  Force  cost  analysts  via  a  mail  survey. 
Experienced  analysts  rated  thirty-five  cost  analysis  subjects  in  terms  of 
usefulness  to  job  performance.  Analysts  were  also  asked  to  identify 
appropriate  educational  methods  for  acquiring  knowledge  In  the  subjects. 
The  result  of  this  effort  is  an  Air  Force  cost  analyst  educational 
requirements  list  by  educational  method. 

Analysis  of  the  survey  data  was  accomplished  utilizing  a  computer  cross¬ 
tabulation  procedure  which  compared  responses  to  each  subject.  The  analy¬ 
sis  used  decision  rules  which  measured  the  central  tendency  of  the 
responses.  Based  on  the  decision  rules  outlined  in  the  methodology 
chapter  subjects  were  recommended  to  be  taught  by  various  educational 
methods. ^sAlso,  written  comments  are  presented  on  how  experienced  cost 
analysts  pldtcelved  the  educational  competence  of  all  Air  Force  cost 
analysts  can  be  Improved. 
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